Messaging SIG

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
m (Add Toshio)
(Brainstorming)
 
(30 intermediate revisions by 13 users not shown)
Line 1: Line 1:
 
== Mission ==
 
== Mission ==
 +
 
Create a Messaging infrastructure within the Fedora Project to facilitate communication, interaction, and integration between services within the Fedora Infrastructure.
 
Create a Messaging infrastructure within the Fedora Project to facilitate communication, interaction, and integration between services within the Fedora Infrastructure.
  
 
== Members ==
 
== Members ==
[[User:jkeating|Jesse Keating]]
+
* [[User:jkeating|Jesse Keating]]
[[User:toshio|Toshio Kuratomi]]
+
* [[User:toshio|Toshio Kuratomi]]
 +
* [[User:mikeb|Mike Bonnet]]
 +
* [[User:ricky|Ricky Zhou]]
 +
* [[User:lmacken|Luke Macken]]
 +
* [[User:johnp|John Palmieri]]
 +
* [[User:herlo|Clint Savage]]
 +
* [[User:bashton|Brennan Ashton]]
 +
* [[User:Ralph|Ralph Bean]]
 +
* [[User:Tflink|Tim Flink]]
 +
* [[User:Jdulaney|John Dulaney]]
 +
 
  
 
== Status ==
 
== Status ==
We're currently in the planning stage of the SIG.  Bringing interested people together to flesh out the who, what, where, when, and how.
+
We have been in a planning stage of the SIG for many years: bringing interested people together to flesh out the who, what, where, when, and how.
 +
 
 +
We started earnest development in Spring of 2012 around [http://github.com/ralphbean/fedmsg fedmsg].  See [http://fedmsg.readthedocs.org/en/latest/status this table] for integration status and [http://fedmsg.readthedocs.org/en/latest/topology this diagram] for the system-wide view.
  
 
== Communication ==
 
== Communication ==
* #fedora-admin and #fedora-devel on Freenode IRC
+
* #fedora-apps and #fedora-admin on Freenode IRC
 +
* Mailing list - https://admin.fedoraproject.org/mailman/listinfo/messaging-sig
 
* [http://jkeating.fedorapeople.org/lfnw-messaging-2009.pdf Slides] from a talk given at Linux Fest Northwest 2009 by Jesse Keating
 
* [http://jkeating.fedorapeople.org/lfnw-messaging-2009.pdf Slides] from a talk given at Linux Fest Northwest 2009 by Jesse Keating
  
 
== Schedule ==
 
== Schedule ==
None yet, need to work out a timeline for implementing a Messaging system.
 
  
== Tasks ==
+
The [http://fedmsg.readthedocs.org/en/latest/#rough-outline-of-stages-of-development-deployment rough outline from the fedmsg docs] break things down into the following 5 major phases:
* Bring people into the SIG
+
 
* Discuss high level goals of the SIG
+
1) Write fedmsg core routines for sending and receiving.  (DONE, in maintenance mode)
* Create a schedule
+
2) Start modifying or instrumenting our apps to use fedmsg to send messages.  (Mostly done, see [http://fedmsg.readthedocs.org/en/latest/status the status table] for what is done and what is left).
* ???
+
3) Consume messages for statistics.  (Done, via [https://github.com/ralphbean/datanommer datnommer].)
* Profit
+
4) Consume messages for user experience.  (Done, via the #fedora-fedmsg irc bot, [https://github.com/lmacken/fedmsg-notify fedmsg-notify], and [http://apps.fedoraproject.org/busmon busmon].
 +
5) Consume messages for interoperability.  (Not started.  Although, [https://fedoraproject.org/wiki/User:Herlo herlo] started work on a consumer to update fama trac tickets in response to FAS fedmsg messages.)
 +
 
 +
== Brainstorming ==
 +
 
 +
Here's a list of things you could work on if you want to help!  It is by no means exhaustive, but is just to get the ball rolling.  Add things here if you have an idea.
 +
 
 +
* Help [https://fedorahosted.org/design-team/ticket/241 design the logo].
 +
* Work on centralizing user notifications preferences by helping with/taking charge of [http://github.com/rossdylan/FUSS FUSS].
 +
* Add hooks to *any* project that doesn't already appear as completed on the [http://fedmsg.readthedocs.org/en/latest/status/ status table].
 +
* Work with RH to get bugzilla messages.
 +
** They'll be providing them via AMQP from qpidd.  We'll need to build a fedmsg service called "fedmsg-amqp-bridge" that receives amqp messages and rebroadcasts them to our zeromq bus.
 +
* Improve the documentation anywhere you see fit!  Is it unclear anywhere?
 +
* Help write a policy for consumers that are acceptable for being hosted in Fedora Infrastructure.
 +
* Start building consumers that respond and react to events on the bus.  Automate our ecosystem!
 +
 
 +
=== Triggered Mirroring ===
 +
 
 +
[https://fedoraproject.org/wiki/User:Ausil dgilmore] mentioned that he knows somebody working on this.
 +
 
 +
# Message when the NetApps snapmirror goes out of sync, due to a write to the master.  Tier 1 mirrors subscribe, and don't pull until next message.
 +
# Message when the NetApps snapmirror is back in sync.  Tier 1 mirrors subscribe, and should pull now.
 +
# Message when each Tier 1 mirror has started syncing.  Tier 2 mirrors subscribe, and don't pull until the next message.
 +
# Message when each Tier 1 mirror has finished syncing.  Tier 2 mirrors subscribe, and should pull from their Tier 1 mirror now.
 +
 
 +
In all cases, having a clue as to which part of the tree has just changed would be useful.  Even a 'directories from this point downward are likely to have changed' would be of benefit.
  
== Resources ==
+
==Naming Scheme==
Links to documentation and project pages for the various tools we'll be using.
+
  
* [http://qpid.apache.org/ QPID project home page]
+
See [http://fedmsg.readthedocs.org/en/latest/overview/#namespace-considerations this section on topic namespace] from the [http://fedmsg.rtfd.org fedmsg docs].  [http://fedmsg.readthedocs.org/en/latest/publishing/ This document on sending messages] might also be helpful.
  
 
[[Category:SIGs]]
 
[[Category:SIGs]]
 
[[Category:Messaging]]
 
[[Category:Messaging]]
 +
[[Category:Fedora special-interest groups]]

Latest revision as of 01:52, 9 October 2012

Contents

[edit] Mission

Create a Messaging infrastructure within the Fedora Project to facilitate communication, interaction, and integration between services within the Fedora Infrastructure.

[edit] Members


[edit] Status

We have been in a planning stage of the SIG for many years: bringing interested people together to flesh out the who, what, where, when, and how.

We started earnest development in Spring of 2012 around fedmsg. See this table for integration status and this diagram for the system-wide view.

[edit] Communication

[edit] Schedule

The rough outline from the fedmsg docs break things down into the following 5 major phases:

1) Write fedmsg core routines for sending and receiving. (DONE, in maintenance mode) 2) Start modifying or instrumenting our apps to use fedmsg to send messages. (Mostly done, see the status table for what is done and what is left). 3) Consume messages for statistics. (Done, via datnommer.) 4) Consume messages for user experience. (Done, via the #fedora-fedmsg irc bot, fedmsg-notify, and busmon. 5) Consume messages for interoperability. (Not started. Although, herlo started work on a consumer to update fama trac tickets in response to FAS fedmsg messages.)

[edit] Brainstorming

Here's a list of things you could work on if you want to help! It is by no means exhaustive, but is just to get the ball rolling. Add things here if you have an idea.

  • Help design the logo.
  • Work on centralizing user notifications preferences by helping with/taking charge of FUSS.
  • Add hooks to *any* project that doesn't already appear as completed on the status table.
  • Work with RH to get bugzilla messages.
    • They'll be providing them via AMQP from qpidd. We'll need to build a fedmsg service called "fedmsg-amqp-bridge" that receives amqp messages and rebroadcasts them to our zeromq bus.
  • Improve the documentation anywhere you see fit! Is it unclear anywhere?
  • Help write a policy for consumers that are acceptable for being hosted in Fedora Infrastructure.
  • Start building consumers that respond and react to events on the bus. Automate our ecosystem!

[edit] Triggered Mirroring

dgilmore mentioned that he knows somebody working on this.

  1. Message when the NetApps snapmirror goes out of sync, due to a write to the master. Tier 1 mirrors subscribe, and don't pull until next message.
  2. Message when the NetApps snapmirror is back in sync. Tier 1 mirrors subscribe, and should pull now.
  3. Message when each Tier 1 mirror has started syncing. Tier 2 mirrors subscribe, and don't pull until the next message.
  4. Message when each Tier 1 mirror has finished syncing. Tier 2 mirrors subscribe, and should pull from their Tier 1 mirror now.

In all cases, having a clue as to which part of the tree has just changed would be useful. Even a 'directories from this point downward are likely to have changed' would be of benefit.

[edit] Naming Scheme

See this section on topic namespace from the fedmsg docs. This document on sending messages might also be helpful.