Aggregate all bundle.yaml | API endpoint

[Ilias Tsoumas]

Hi Merlijn,

First of all thank you for your quick and essential answer!

Reply inline

Great to hear you're doing cool stuff with Juju! I recently asked a similar

question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If you
want, I can send it to you in private (it's accepted but not published yet).

Of course. I would like to read your paper. Also me and a colleague work on
performance estimations of app components (=charms). If I have suspected
properly your paper is related with performance analysis/profiling thus it
will be inspiration for us.

https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md

I will check it.

https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle

Thank you! I hope to be feasible to pull all bundle.yaml with one
query/request.

BR,
Ilias

On Mon, Jun 4, 2018 at 3:22 PM Merlijn Sebrechts <

Hi Ilias
Great to hear you're doing cool stuff with Juju! I recently asked a
similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If you
want, I can send it to you in private (it's accepted but not published yet).
https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive/bundle.yaml
https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle
Cleaning up the data will be the hard part probably, since there are a lot
of unused/broken bundles in the store. I've used the downloads/month metric
to figure out what charms are actually still used.

Hello all,
I have designed and I am developing a novel recommender system which will
provide recommendations at application graph (=bundle) composition time.
Now I need a real graphs dataset. I think that all juju bundles of
components is a good real dataset. Are there some how to aggregate all
these "bundle.yaml" files? Are there some api endpoint?
This work is funding from the European Union’s Horizon 2020 research and
innovation program under grant agreement No 761898 project Matilda.
Kind regards,
Ilias Tsoumas
--
Juju mailing list
https://lists.ubuntu.com/mailman/listinfo/juju

[Ilias Tsoumas]

Hello Merlijn and all,

I developed a python script which fetching all the bundle.yml.
Here is the repo: https://github.com/itsoum/BundlesFetcher
Merlijn special thanks for the help about the API.

Now, there rise a new "problem". As I had mentioned in the first-post above
I fetch bundles for the relations between nodes(charms). But there isn't an
obvious annotation technique about how relations are described inside each
bundle.yaml.
Check some different examples below
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml

https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/web-in-a-box-9.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>

https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/hadoop-spark-1.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>

For example in the first:
relations:
- - haproxy:reverseproxy
- wordpress:website
- - wordpress:cache
- memcached:cache
- - wordpress:db
- mariadb:db
- - mariadb-slave:slave
- mariadb:master
- - nagios:nagios
- wordpress:juju-info which is the topology of application graph. The ':'
isn't denotes the edge between two charms? If yes, why - wordpress:db and -
mariadb:db?
There us someone who knows the annotation method of relations? Could you
explain?

BR,
Ilias

Post by Ilias Tsoumas
Hi Merlijn,
First of all thank you for your quick and essential answer!
Reply inline
Great to hear you're doing cool stuff with Juju! I recently asked a

similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If you
want, I can send it to you in private (it's accepted but not published yet).

Of course. I would like to read your paper. Also me and a colleague work
on performance estimations of app components (=charms). If I have suspected
properly your paper is related with performance analysis/profiling thus it
will be inspiration for us.

https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md

I will check it.

https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle

Thank you! I hope to be feasible to pull all bundle.yaml with one
query/request.
BR,
Ilias
On Mon, Jun 4, 2018 at 3:22 PM Merlijn Sebrechts <

Hi Ilias
Great to hear you're doing cool stuff with Juju! I recently asked a
similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If you
want, I can send it to you in private (it's accepted but not published yet).
https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive/bundle.yaml
https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle
Cleaning up the data will be the hard part probably, since there are a
lot of unused/broken bundles in the store. I've used the downloads/month
metric to figure out what charms are actually still used.

Hello all,
I have designed and I am developing a novel recommender system which
will provide recommendations at application graph (=bundle) composition
time. Now I need a real graphs dataset. I think that all juju bundles of
components is a good real dataset. Are there some how to aggregate all
these "bundle.yaml" files? Are there some api endpoint?
This work is funding from the European Union’s Horizon 2020 research and
innovation program under grant agreement No 761898 project Matilda.
Kind regards,
Ilias Tsoumas
--
Juju mailing list
https://lists.ubuntu.com/mailman/listinfo/juju

[Merlijn Sebrechts]

The format is

- - <application-name1>:<endpoint-name1>
- <application-name2>:<endpoint-name2>

Where application-name is the name of the _instance_ of the charm (as
defined in the applications section of bundle.yaml) and endpoint-name is
the name of the endpoint (requires/provides from metadata.yaml) that is
used to connect the two nodes.

Endpoint-name is optional if only one type of relation is possible between
the nodes (a relation is established between a required endpoint of one
node and a provides endpoint of the same interface type of the other node).
The : is not present if the endpoint name is not specified.

So the relations section is a 2 dimensional list, the first dimension is an
array of all the relations and the second dimension specifies which
endpoints of which nodes that relation connects.

And finally, the applications section is called "services" in some bundles.

I think that covers most of the strange quirks, let me know if you have
more questions.

Post by Ilias Tsoumas
Hello Merlijn and all,
I developed a python script which fetching all the bundle.yml.
Here is the repo: https://github.com/itsoum/BundlesFetcher
Merlijn special thanks for the help about the API.
Now, there rise a new "problem". As I had mentioned in the first-post
above I fetch bundles for the relations between nodes(charms). But there
isn't an obvious annotation technique about how relations are described
inside each bundle.yaml.
Check some different examples below
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/web-in-a-box-9.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/hadoop-spark-1.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
- - haproxy:reverseproxy
- wordpress:website
- - wordpress:cache
- memcached:cache
- - wordpress:db
- mariadb:db
- - mariadb-slave:slave
- mariadb:master
- - nagios:nagios
- wordpress:juju-info which is the topology of application graph. The ':'
isn't denotes the edge between two charms? If yes, why - wordpress:db and -
mariadb:db?
There us someone who knows the annotation method of relations? Could you
explain?
BR,
Ilias

Post by Ilias Tsoumas
Hi Merlijn,
First of all thank you for your quick and essential answer!
Reply inline
Great to hear you're doing cool stuff with Juju! I recently asked a

similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If you
want, I can send it to you in private (it's accepted but not published yet).

Of course. I would like to read your paper. Also me and a colleague work
on performance estimations of app components (=charms). If I have suspected
properly your paper is related with performance analysis/profiling thus it
will be inspiration for us.

https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md

I will check it.
Here is an Jupyter notebook I use to get metrics of charm and layer

https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle

Thank you! I hope to be feasible to pull all bundle.yaml with one
query/request.
BR,
Ilias
On Mon, Jun 4, 2018 at 3:22 PM Merlijn Sebrechts <

Hi Ilias
Great to hear you're doing cool stuff with Juju! I recently asked a
similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If you
want, I can send it to you in private (it's accepted but not published yet).
https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive/bundle.yaml
https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle
Cleaning up the data will be the hard part probably, since there are a
lot of unused/broken bundles in the store. I've used the downloads/month
metric to figure out what charms are actually still used.

Hello all,
I have designed and I am developing a novel recommender system which
will provide recommendations at application graph (=bundle) composition
time. Now I need a real graphs dataset. I think that all juju bundles of
components is a good real dataset. Are there some how to aggregate all
these "bundle.yaml" files? Are there some api endpoint?
This work is funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement No 761898 project Matilda.
Kind regards,
Ilias Tsoumas
--
Juju mailing list
https://lists.ubuntu.com/mailman/listinfo/juju

[Ilias Tsoumas]

Merlijn thank you for your reply. It was enlightening about the relations
of charms.
Only a comment till now, is each bundle a directed acyclic graph (DAG)?


On Tue, Jun 26, 2018 at 7:10 PM Merlijn Sebrechts <

Post by Merlijn Sebrechts
The format is
- - <application-name1>:<endpoint-name1>
- <application-name2>:<endpoint-name2>
Where application-name is the name of the _instance_ of the charm (as
defined in the applications section of bundle.yaml) and endpoint-name is
the name of the endpoint (requires/provides from metadata.yaml) that is
used to connect the two nodes.
Endpoint-name is optional if only one type of relation is possible between
the nodes (a relation is established between a required endpoint of one
node and a provides endpoint of the same interface type of the other node).
The : is not present if the endpoint name is not specified.
So the relations section is a 2 dimensional list, the first dimension is
an array of all the relations and the second dimension specifies which
endpoints of which nodes that relation connects.
And finally, the applications section is called "services" in some bundles.
I think that covers most of the strange quirks, let me know if you have
more questions.

Post by Ilias Tsoumas
Hello Merlijn and all,
I developed a python script which fetching all the bundle.yml.
Here is the repo: https://github.com/itsoum/BundlesFetcher
Merlijn special thanks for the help about the API.
Now, there rise a new "problem". As I had mentioned in the first-post
above I fetch bundles for the relations between nodes(charms). But there
isn't an obvious annotation technique about how relations are described
inside each bundle.yaml.
Check some different examples below
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/web-in-a-box-9.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/hadoop-spark-1.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
- - haproxy:reverseproxy
- wordpress:website
- - wordpress:cache
- memcached:cache
- - wordpress:db
- mariadb:db
- - mariadb-slave:slave
- mariadb:master
- - nagios:nagios
- wordpress:juju-info which is the topology of application graph. The
':' isn't denotes the edge between two charms? If yes, why - wordpress:db
and - mariadb:db?
There us someone who knows the annotation method of relations? Could you
explain?
BR,
Ilias

Post by Ilias Tsoumas
Hi Merlijn,
First of all thank you for your quick and essential answer!
Reply inline
Great to hear you're doing cool stuff with Juju! I recently asked a

similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
you want, I can send it to you in private (it's accepted but not published
yet).

Of course. I would like to read your paper. Also me and a colleague work
on performance estimations of app components (=charms). If I have suspected
properly your paper is related with performance analysis/profiling thus it
will be inspiration for us.

https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md

I will check it.
Here is an Jupyter notebook I use to get metrics of charm and layer

https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle

Thank you! I hope to be feasible to pull all bundle.yaml with one
query/request.
BR,
Ilias
On Mon, Jun 4, 2018 at 3:22 PM Merlijn Sebrechts <

Hi Ilias
Great to hear you're doing cool stuff with Juju! I recently asked a
similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
you want, I can send it to you in private (it's accepted but not published
yet).
https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive/bundle.yaml
https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle
Cleaning up the data will be the hard part probably, since there are a
lot of unused/broken bundles in the store. I've used the downloads/month
metric to figure out what charms are actually still used.

Hello all,
I have designed and I am developing a novel recommender system which
will provide recommendations at application graph (=bundle) composition
time. Now I need a real graphs dataset. I think that all juju bundles of
components is a good real dataset. Are there some how to aggregate all
these "bundle.yaml" files? Are there some api endpoint?
This work is funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement No 761898 project Matilda.
Kind regards,
Ilias Tsoumas
--
Juju mailing list
https://lists.ubuntu.com/mailman/listinfo/juju

[Merlijn Sebrechts]

There is nothing preventing directed cycles in the graph, although I'm not
sure how common they are in real bundles.

Post by Ilias Tsoumas
Merlijn thank you for your reply. It was enlightening about the relations
of charms.
Only a comment till now, is each bundle a directed acyclic graph (DAG)?
On Tue, Jun 26, 2018 at 7:10 PM Merlijn Sebrechts <

Post by Merlijn Sebrechts
The format is
- - <application-name1>:<endpoint-name1>
- <application-name2>:<endpoint-name2>
Where application-name is the name of the _instance_ of the charm (as
defined in the applications section of bundle.yaml) and endpoint-name is
the name of the endpoint (requires/provides from metadata.yaml) that is
used to connect the two nodes.
Endpoint-name is optional if only one type of relation is possible
between the nodes (a relation is established between a required endpoint of
one node and a provides endpoint of the same interface type of the other
node). The : is not present if the endpoint name is not specified.
So the relations section is a 2 dimensional list, the first dimension is
an array of all the relations and the second dimension specifies which
endpoints of which nodes that relation connects.
And finally, the applications section is called "services" in some bundles.
I think that covers most of the strange quirks, let me know if you have
more questions.

Post by Ilias Tsoumas
Hello Merlijn and all,
I developed a python script which fetching all the bundle.yml.
Here is the repo: https://github.com/itsoum/BundlesFetcher
Merlijn special thanks for the help about the API.
Now, there rise a new "problem". As I had mentioned in the first-post
above I fetch bundles for the relations between nodes(charms). But there
isn't an obvious annotation technique about how relations are described
inside each bundle.yaml.
Check some different examples below
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/web-in-a-box-9.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/hadoop-spark-1.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
- - haproxy:reverseproxy
- wordpress:website
- - wordpress:cache
- memcached:cache
- - wordpress:db
- mariadb:db
- - mariadb-slave:slave
- mariadb:master
- - nagios:nagios
- wordpress:juju-info which is the topology of application graph. The
':' isn't denotes the edge between two charms? If yes, why - wordpress:db
and - mariadb:db?
There us someone who knows the annotation method of relations? Could you
explain?
BR,
Ilias

Post by Ilias Tsoumas
Hi Merlijn,
First of all thank you for your quick and essential answer!
Reply inline
Great to hear you're doing cool stuff with Juju! I recently asked a

similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
you want, I can send it to you in private (it's accepted but not published
yet).

Of course. I would like to read your paper. Also me and a colleague
work on performance estimations of app components (=charms). If I have
suspected properly your paper is related with performance
analysis/profiling thus it will be inspiration for us.

https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md

I will check it.
Here is an Jupyter notebook I use to get metrics of charm and layer

https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle

Thank you! I hope to be feasible to pull all bundle.yaml with one
query/request.
BR,
Ilias
On Mon, Jun 4, 2018 at 3:22 PM Merlijn Sebrechts <

Hi Ilias
Great to hear you're doing cool stuff with Juju! I recently asked a
similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
you want, I can send it to you in private (it's accepted but not published
yet).
https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive/bundle.yaml
https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle
Cleaning up the data will be the hard part probably, since there are a
lot of unused/broken bundles in the store. I've used the downloads/month
metric to figure out what charms are actually still used.

Hello all,
I have designed and I am developing a novel recommender system which
will provide recommendations at application graph (=bundle) composition
time. Now I need a real graphs dataset. I think that all juju bundles of
components is a good real dataset. Are there some how to aggregate all
these "bundle.yaml" files? Are there some api endpoint?
This work is funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement No 761898 project Matilda.
Kind regards,
Ilias Tsoumas
--
Juju mailing list
https://lists.ubuntu.com/mailman/listinfo/juju

[Ilias Tsoumas]

Ok, it is clear.
Also, how I identify the "direction" of relationship? It is denoted with
order of the dashes?
*- - <application-name1>:<endpoint-name1>*
* - <application-name2>:<endpoint-name2>*
imply that the data-flow from *<application-name1>* is directed in
*<application-name2>*
and
*- - <application-name2>:<endpoint-name2>*
* - <application-name1>:<endpoint-name1>*
imply that the data-flow from *<application-name2>* is directed in
*<application-name1>*

On Thu, Jul 5, 2018 at 5:10 PM Merlijn Sebrechts <

Post by Merlijn Sebrechts
There is nothing preventing directed cycles in the graph, although I'm not
sure how common they are in real bundles.

Post by Ilias Tsoumas
Merlijn thank you for your reply. It was enlightening about the relations
of charms.
Only a comment till now, is each bundle a directed acyclic graph (DAG)?
On Tue, Jun 26, 2018 at 7:10 PM Merlijn Sebrechts <

Post by Merlijn Sebrechts
The format is
- - <application-name1>:<endpoint-name1>
- <application-name2>:<endpoint-name2>
Where application-name is the name of the _instance_ of the charm (as
defined in the applications section of bundle.yaml) and endpoint-name is
the name of the endpoint (requires/provides from metadata.yaml) that is
used to connect the two nodes.
Endpoint-name is optional if only one type of relation is possible
between the nodes (a relation is established between a required endpoint of
one node and a provides endpoint of the same interface type of the other
node). The : is not present if the endpoint name is not specified.
So the relations section is a 2 dimensional list, the first dimension is
an array of all the relations and the second dimension specifies which
endpoints of which nodes that relation connects.
And finally, the applications section is called "services" in some bundles.
I think that covers most of the strange quirks, let me know if you have
more questions.

Post by Ilias Tsoumas
Hello Merlijn and all,
I developed a python script which fetching all the bundle.yml.
Here is the repo: https://github.com/itsoum/BundlesFetcher
Merlijn special thanks for the help about the API.
Now, there rise a new "problem". As I had mentioned in the first-post
above I fetch bundles for the relations between nodes(charms). But there
isn't an obvious annotation technique about how relations are described
inside each bundle.yaml.
Check some different examples below
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/web-in-a-box-9.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/hadoop-spark-1.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
- - haproxy:reverseproxy
- wordpress:website
- - wordpress:cache
- memcached:cache
- - wordpress:db
- mariadb:db
- - mariadb-slave:slave
- mariadb:master
- - nagios:nagios
- wordpress:juju-info which is the topology of application graph. The
':' isn't denotes the edge between two charms? If yes, why - wordpress:db
and - mariadb:db?
There us someone who knows the annotation method of relations? Could
you explain?
BR,
Ilias

Post by Ilias Tsoumas
Hi Merlijn,
First of all thank you for your quick and essential answer!
Reply inline
Great to hear you're doing cool stuff with Juju! I recently asked a

similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
you want, I can send it to you in private (it's accepted but not published
yet).

Of course. I would like to read your paper. Also me and a colleague
work on performance estimations of app components (=charms). If I have
suspected properly your paper is related with performance
analysis/profiling thus it will be inspiration for us.

https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md

I will check it.
Here is an Jupyter notebook I use to get metrics of charm and layer

https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle

Thank you! I hope to be feasible to pull all bundle.yaml with one
query/request.
BR,
Ilias
On Mon, Jun 4, 2018 at 3:22 PM Merlijn Sebrechts <

Hi Ilias
Great to hear you're doing cool stuff with Juju! I recently asked a
similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
you want, I can send it to you in private (it's accepted but not published
yet).
https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive/bundle.yaml
https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle
Cleaning up the data will be the hard part probably, since there are
a lot of unused/broken bundles in the store. I've used the downloads/month
metric to figure out what charms are actually still used.

Hello all,
I have designed and I am developing a novel recommender system which
will provide recommendations at application graph (=bundle) composition
time. Now I need a real graphs dataset. I think that all juju bundles of
components is a good real dataset. Are there some how to aggregate all
these "bundle.yaml" files? Are there some api endpoint?
This work is funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement No 761898 project Matilda.
Kind regards,
Ilias Tsoumas
--
Juju mailing list
https://lists.ubuntu.com/mailman/listinfo/juju

[Marco Ceppi]

Data flow in juju is bi-directional the order does not image or change
anything in the relation

Post by Ilias Tsoumas
Ok, it is clear.
Also, how I identify the "direction" of relationship? It is denoted with
order of the dashes?
*- - <application-name1>:<endpoint-name1>*
* - <application-name2>:<endpoint-name2>*
imply that the data-flow from *<application-name1>* is directed in
*<application-name2>*
and
*- - <application-name2>:<endpoint-name2>*
* - <application-name1>:<endpoint-name1>*
imply that the data-flow from *<application-name2>* is directed in
*<application-name1>*
On Thu, Jul 5, 2018 at 5:10 PM Merlijn Sebrechts <

Post by Merlijn Sebrechts
There is nothing preventing directed cycles in the graph, although I'm
not sure how common they are in real bundles.

Post by Ilias Tsoumas
Merlijn thank you for your reply. It was enlightening about the
relations of charms.
Only a comment till now, is each bundle a directed acyclic graph (DAG)?
On Tue, Jun 26, 2018 at 7:10 PM Merlijn Sebrechts <

Post by Merlijn Sebrechts
The format is
- - <application-name1>:<endpoint-name1>
- <application-name2>:<endpoint-name2>
Where application-name is the name of the _instance_ of the charm (as
defined in the applications section of bundle.yaml) and endpoint-name is
the name of the endpoint (requires/provides from metadata.yaml) that is
used to connect the two nodes.
Endpoint-name is optional if only one type of relation is possible
between the nodes (a relation is established between a required endpoint of
one node and a provides endpoint of the same interface type of the other
node). The : is not present if the endpoint name is not specified.
So the relations section is a 2 dimensional list, the first dimension
is an array of all the relations and the second dimension specifies which
endpoints of which nodes that relation connects.
And finally, the applications section is called "services" in some bundles.
I think that covers most of the strange quirks, let me know if you have
more questions.

Post by Ilias Tsoumas
Hello Merlijn and all,
I developed a python script which fetching all the bundle.yml.
Here is the repo: https://github.com/itsoum/BundlesFetcher
Merlijn special thanks for the help about the API.
Now, there rise a new "problem". As I had mentioned in the first-post
above I fetch bundles for the relations between nodes(charms). But there
isn't an obvious annotation technique about how relations are described
inside each bundle.yaml.
Check some different examples below
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/web-in-a-box-9.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/hadoop-spark-1.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
- - haproxy:reverseproxy
- wordpress:website
- - wordpress:cache
- memcached:cache
- - wordpress:db
- mariadb:db
- - mariadb-slave:slave
- mariadb:master
- - nagios:nagios
- wordpress:juju-info which is the topology of application graph. The
':' isn't denotes the edge between two charms? If yes, why - wordpress:db
and - mariadb:db?
There us someone who knows the annotation method of relations? Could
you explain?
BR,
Ilias

Post by Ilias Tsoumas
Hi Merlijn,
First of all thank you for your quick and essential answer!
Reply inline
Great to hear you're doing cool stuff with Juju! I recently asked a

similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
you want, I can send it to you in private (it's accepted but not published
yet).

Of course. I would like to read your paper. Also me and a colleague
work on performance estimations of app components (=charms). If I have
suspected properly your paper is related with performance
analysis/profiling thus it will be inspiration for us.

https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md

I will check it.
Here is an Jupyter notebook I use to get metrics of charm and layer

https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle

Thank you! I hope to be feasible to pull all bundle.yaml with one
query/request.
BR,
Ilias
On Mon, Jun 4, 2018 at 3:22 PM Merlijn Sebrechts <

Hi Ilias
Great to hear you're doing cool stuff with Juju! I recently asked a
similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
you want, I can send it to you in private (it's accepted but not published
yet).
https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive/bundle.yaml
https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle
Cleaning up the data will be the hard part probably, since there are
a lot of unused/broken bundles in the store. I've used the downloads/month
metric to figure out what charms are actually still used.

Hello all,
I have designed and I am developing a novel recommender system
which will provide recommendations at application graph (=bundle)
composition time. Now I need a real graphs dataset. I think that all juju
bundles of components is a good real dataset. Are there some how to
aggregate all these "bundle.yaml" files? Are there some api endpoint?
This work is funding from the European Union’s Horizon 2020
research and innovation program under grant agreement No 761898 project
Matilda.
Kind regards,
Ilias Tsoumas
--
Juju mailing list
https://lists.ubuntu.com/mailman/listinfo/juju

--

Juju mailing list
https://lists.ubuntu.com/mailman/listinfo/juju

[Mark Shuttleworth]

Post by Marco Ceppi
Data flow in juju is bi-directional the order does not image or change
anything in the relation

Data flow is bi-directional, but my understanding is that a Juju
interface has the idea of a provider and a consumer, which I think is
the directionality Ilias is looking for.

Mark

[Ilias Tsoumas]

Post by Mark Shuttleworth
a Juju
interface has the idea of a provider and a consumer, which I think is
the directionality Ilias is looking for.

Hello Mark,

For the sake of clarification, let's try to explain my thoughts. I want to
understand if I can use juju's bundles as directed (acyclic) graphs. Is
there the information about direction? If no, do you propose to use the
bundles as undirected graphs? Otherwise, where I can find the
aforementioned directionality?

Post by Mark Shuttleworth
If you want to know the direction of a relationship, you need to look at
the endpoints in the metadata.yaml of both charms and see which is the
provides side and which is the requires side.

In this case, I should do an api call for each relation in order to check
if the :endpoint part is under the requires or provides tag of metada.yaml.

Thank you,
Ilias

Post by Mark Shuttleworth

Post by Marco Ceppi
Data flow in juju is bi-directional the order does not image or change
anything in the relation

Data flow is bi-directional, but my understanding is that a Juju
interface has the idea of a provider and a consumer, which I think is
the directionality Ilias is looking for.
Mark

[Merlijn Sebrechts]

As Marco said; The direction is purely semantical; it provides some
information about what the relation represents but it doesn't specify the
direction of a data flow or something else. A relation always connects a
"provides" endpoint with a "requires" endpoint, you can see this as the
relation having a direction. If you want to know the direction of a
relationship, you need to look at the endpoints in the metadata.yaml of
both charms and see which is the provides side and which is the requires
side.

Also note that relations denote an operations-level (management-level)
dependency. This does not necessarily mean that there is data flowing
between the *applications* themselves, only that the charms who manage the
applications share data.

Post by Ilias Tsoumas
Ok, it is clear.
Also, how I identify the "direction" of relationship? It is denoted with
order of the dashes?
*- - <application-name1>:<endpoint-name1>*
* - <application-name2>:<endpoint-name2>*
imply that the data-flow from *<application-name1>* is directed in
*<application-name2>*
and
*- - <application-name2>:<endpoint-name2>*
* - <application-name1>:<endpoint-name1>*
imply that the data-flow from *<application-name2>* is directed in
*<application-name1>*
On Thu, Jul 5, 2018 at 5:10 PM Merlijn Sebrechts <

Post by Merlijn Sebrechts
There is nothing preventing directed cycles in the graph, although I'm
not sure how common they are in real bundles.

Post by Ilias Tsoumas
Merlijn thank you for your reply. It was enlightening about the
relations of charms.
Only a comment till now, is each bundle a directed acyclic graph (DAG)?
On Tue, Jun 26, 2018 at 7:10 PM Merlijn Sebrechts <

Post by Merlijn Sebrechts
The format is
- - <application-name1>:<endpoint-name1>
- <application-name2>:<endpoint-name2>
Where application-name is the name of the _instance_ of the charm (as
defined in the applications section of bundle.yaml) and endpoint-name is
the name of the endpoint (requires/provides from metadata.yaml) that is
used to connect the two nodes.
Endpoint-name is optional if only one type of relation is possible
between the nodes (a relation is established between a required endpoint of
one node and a provides endpoint of the same interface type of the other
node). The : is not present if the endpoint name is not specified.
So the relations section is a 2 dimensional list, the first dimension
is an array of all the relations and the second dimension specifies which
endpoints of which nodes that relation connects.
And finally, the applications section is called "services" in some bundles.
I think that covers most of the strange quirks, let me know if you have
more questions.

Post by Ilias Tsoumas
Hello Merlijn and all,
I developed a python script which fetching all the bundle.yml.
Here is the repo: https://github.com/itsoum/BundlesFetcher
Merlijn special thanks for the help about the API.
Now, there rise a new "problem". As I had mentioned in the first-post
above I fetch bundles for the relations between nodes(charms). But there
isn't an obvious annotation technique about how relations are described
inside each bundle.yaml.
Check some different examples below
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/web-in-a-box-9.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/hadoop-spark-1.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
- - haproxy:reverseproxy
- wordpress:website
- - wordpress:cache
- memcached:cache
- - wordpress:db
- mariadb:db
- - mariadb-slave:slave
- mariadb:master
- - nagios:nagios
- wordpress:juju-info which is the topology of application graph. The
':' isn't denotes the edge between two charms? If yes, why - wordpress:db
and - mariadb:db?
There us someone who knows the annotation method of relations? Could
you explain?
BR,
Ilias

Post by Ilias Tsoumas
Hi Merlijn,
First of all thank you for your quick and essential answer!
Reply inline
Great to hear you're doing cool stuff with Juju! I recently asked a

similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
you want, I can send it to you in private (it's accepted but not published
yet).

Of course. I would like to read your paper. Also me and a colleague
work on performance estimations of app components (=charms). If I have
suspected properly your paper is related with performance
analysis/profiling thus it will be inspiration for us.

https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md

I will check it.
Here is an Jupyter notebook I use to get metrics of charm and layer

https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle

Thank you! I hope to be feasible to pull all bundle.yaml with one
query/request.
BR,
Ilias
On Mon, Jun 4, 2018 at 3:22 PM Merlijn Sebrechts <

Hi Ilias
Great to hear you're doing cool stuff with Juju! I recently asked a
similar question for a paper I've written for CLOUD 2018
https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
you want, I can send it to you in private (it's accepted but not published
yet).
https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive
https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive/bundle.yaml
https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
It should be easy to modify that code to pull info about the bundles. As an
https://api.jujucharms.com/charmstore/v5/list?type=bundle
Cleaning up the data will be the hard part probably, since there are
a lot of unused/broken bundles in the store. I've used the downloads/month
metric to figure out what charms are actually still used.

Hello all,
I have designed and I am developing a novel recommender system
which will provide recommendations at application graph (=bundle)
composition time. Now I need a real graphs dataset. I think that all juju
bundles of components is a good real dataset. Are there some how to
aggregate all these "bundle.yaml" files? Are there some api endpoint?
This work is funding from the European Union’s Horizon 2020
research and innovation program under grant agreement No 761898 project
Matilda.
Kind regards,
Ilias Tsoumas
--
Juju mailing list
https://lists.ubuntu.com/mailman/listinfo/juju