Serverless Model Type

[James Beedy]

How do people feel about Juju supporting a serverless model type, and
serverless charm?

Some innovative wrappers popping up that facilitate deploying lambda
functions:
- `serverless deploy` - https://serverless.com/
- `zappa deploy` - https://github.com/Miserlou/Zappa

I feel like Juju has the proper constructs needed to support a SL model
type, and a special SL type charm.

Just an idea, thought I would throw it out there....

Thoughts?

[Merlijn Sebrechts]

+1

With some care, one solution could be used for both containers, serverless
functions and more. We just need two new types of charms:

- A "platform" charm which connects to the serverless platform and deploys
the function.
- A "job" charm which describes the function. This is basically a bundle
of metadata.

Connecting the `platform` to the `job` charm will cause a `job-joined` hook
in the platform charm, which can then use `job-get` to get the metadata
describing the function and deploy the function on the platform. This way,
you empower the community to create new "platform" charms for whichever
platforms they see fit: AWS lambda, Kubernetes, Heroku and even Apache
Spark.

- Juju doesn't need to know the details of each platform because the
connection to the platform is part of the charm.
- Juju doesn't need to know the details of each "job" type, because the
"job" charm is just a bunch of metadata.

Now add the ability for "job" charms to optionally include relation hooks,
and you can seamlessly connect serverless functions to your existing
infrastructure. Or connect Apache Spark jobs to your existing
infrastructure.

We've created a number of prototypes of this to model and deploy Kubernetes
and Apache Storm topologies in Juju. We used normal charms and relations
for our "platform" and "job" charms. This works really wel, the only issue
is that deploying an entire Juju Agent is waaaay too much overhead for
modeling a single Docker container, so we would be really happy if Juju
could introduce the idea of "job" charms.



Kind regards
Merlijn

Post by James Beedy
How do people feel about Juju supporting a serverless model type, and
serverless charm?
Some innovative wrappers popping up that facilitate deploying lambda
- `serverless deploy` - https://serverless.com/
- `zappa deploy` - https://github.com/Miserlou/Zappa
I feel like Juju has the proper constructs needed to support a SL model
type, and a special SL type charm.
Just an idea, thought I would throw it out there....
Thoughts?
--
Juju-dev mailing list
Modify settings or unsubscribe at: https://lists.ubuntu.com/
mailman/listinfo/juju-dev

[James Beedy]

Merlijn,

That is great insight, thank you!

The CAAS stuff incorporates some of these concepts already (I think), but
is pretty k8s specific while in the dev phase from my understanding.

Abstracting the CAAS charm and model to facilitate a more general
platform/job architecture so as it could be extended to better support
multiple platforms - this sounds like the golden ticket.


~James


On Thu, Nov 16, 2017 at 8:32 AM, Merlijn Sebrechts <

Post by Merlijn Sebrechts
+1
With some care, one solution could be used for both containers, serverless
- A "platform" charm which connects to the serverless platform and
deploys the function.
- A "job" charm which describes the function. This is basically a bundle
of metadata.
Connecting the `platform` to the `job` charm will cause a `job-joined`
hook in the platform charm, which can then use `job-get` to get the
metadata describing the function and deploy the function on the platform.
This way, you empower the community to create new "platform" charms for
whichever platforms they see fit: AWS lambda, Kubernetes, Heroku and even
Apache Spark.
- Juju doesn't need to know the details of each platform because the
connection to the platform is part of the charm.
- Juju doesn't need to know the details of each "job" type, because the
"job" charm is just a bunch of metadata.
Now add the ability for "job" charms to optionally include relation hooks,
and you can seamlessly connect serverless functions to your existing
infrastructure. Or connect Apache Spark jobs to your existing
infrastructure.
We've created a number of prototypes of this to model and deploy
Kubernetes and Apache Storm topologies in Juju. We used normal charms and
relations for our "platform" and "job" charms. This works really wel, the
only issue is that deploying an entire Juju Agent is waaaay too much
overhead for modeling a single Docker container, so we would be really
happy if Juju could introduce the idea of "job" charms.
Kind regards
Merlijn

Post by James Beedy
How do people feel about Juju supporting a serverless model type, and
serverless charm?
Some innovative wrappers popping up that facilitate deploying lambda
- `serverless deploy` - https://serverless.com/
- `zappa deploy` - https://github.com/Miserlou/Zappa
I feel like Juju has the proper constructs needed to support a SL model
type, and a special SL type charm.
Just an idea, thought I would throw it out there....
Thoughts?
--
Juju-dev mailing list
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm
an/listinfo/juju-dev

[Mark Shuttleworth]

Yes, this is a very natural evolution for Juju. I think the concept of
'scale' is fundamentally different in serverless environments or models,
but many of the other concepts in Juju map neatly and we can plan to get
there. I also think that the different serverless implementations are
much less consistent with one another than, say, different approaches to
the orchestration of docker containers. That means that we're probably
better of waiting a while till the serverless world shakes out further,
while continuing to strengthen the underlying concepts in Juju in the
realm of machines and containers.

Mark