Support
Help Save Reptile!
Navigation

Essentials

Installation

Developers

P2P (content distribution)

Search Infrastructure

Services

Proposals

Resources

Overview

The current (0.5.0) Reptile subscription mechanism is basically broken. It has taken us far from 0.0.3 but now needs to be rewritten.

There are 2 major isssues:

  1. Subscriptions don't use the database.
  2. New subscriptions don't fetch synchronously. This means we get an error on the first request and we have to wait until it is updated on disk.

Each issue is related to the other and we have to make this change at once and make sure we update each subsystem at the same time or this may break Reptile.


DONE: Subscriptions don't use the database

Right now we keep subscriptions in /xml/users/default/subscriptions.xml.

This means that every time we want to update the subscriptions we need to apply a stylesheet instead of just using the db.

The solution is to migrate to a CHANNEL table (with a SUBSCRIBED column) and update this when necessary. Each subscription should have an associated CATEGORY from a CATEGORY table.

We also need to make sure we modernize the "Edit Channel" control. It should be renamed "Subscribe to channel", include help, have an icon, and only display the subscription info.

Dependencies on the current system

The urn:subscriptions sequence should be reworked to support the DB. We should organize layers around subscription categories. It should also be renamed urn:home (make sure to do a global search/replace here).

The new urn:home sequence should use the DB to serialize out layout XML. We should just serialize <layout:content> elements because we need to combine this with another XML file for required layers (unread articles, etc). The default layer should be the 'Unread Articles' layer.

When serializing subscriptions onto a layer. We should go from left to right alphabetically so that we get an even distribution. (NOTE: we need to remove the 'side' control on subscription).


Default subscriptions

DONE: All default subscriptiosn should be installed in DB with a default-subscriptions.sql file. This should first create the necessary categories and then link to them with new subscriptions (make sure to use the FK correctly).


Serializing subscriptions to layout.

We need a new Xalan extension. SubscriptionLayoutExtension that will use the Channel table to provide layout markup. This will be used by misc sequence to view categories.

This should include 'layers' and 'entries' extension elements so that I can get a 'layers' section an then a list of entries.



New subscriptions don't fetch synchronously

This is a major problem. Right now we are using the on-disk RSS. If for some reason the XML is invalid, we can't view a channel and we get a TON of errors. If we would view RSS channels right from the DB we wouldn't have this problem. An RSS channel could break and we could go about our business (without affecting the UI) until it returned.

The other part of this is that channels aren't fetched synchronously. If we need to fetch a channel we first throw an Exception and THEN, after the user refreshes (sometimes multiple times) do we show the channel. This is VERY ugly and needs to be rewritten.

Solution

Rewrite the http://schemas.openprivacy.org/reptile/control/subset to:

  1. Run a search on the ArticleSearchProvider for this channel and return the top 10 (this should be a setting 5, 10, 15) sorted by date descending.
  2. Serialize the first page
  3. XSLT this to RSS 1.0
  4. Pass this to rss-1.0.xsl

Internally, the first step should use a ChannelRenderExtension. Its main job is to output RSS 1.0 when given a channel URL. How it does this is dependant on whether this channel is already subscribed.

  • When asked to render a channel it should first check to see if this is subscribed. If it isn't, we should use the proxy to get the RSS content and use the content-transformer to produce RSS 1.0.
  • If it is subscribed, we should use the DB to produce RSS 1.0.

XSLT sequence

The following should be used as the new XSLT sequence for rendering RSS (as a control) in the new system.

  1. bootstrap.xml: starts the whole process rolling.
  2. channel-render.xsl: uses the ChannelRenderExtension to output RSS 1.0
  3. rss-1.0.xsl: rewrites the RSS 1.0 as an XHTML control

Design of ChannelRenderExtension

The channel render extension is responsible for providing RSS 1.0 of a channel in the fastest and most reliable means possible.

For subscribed content, we assume it is being updated in the background and its articles are making it into the DB. We then run a request to pull the most recent 10 articles from the DB. We then bypass the SearchSerializer (for speed reasons) and use JDOM to produce RSS 1.0 from the results (similar to the way SearchSerializer works).

For remote content we need use the Panther proxy to wait until we have the content local. If it is already RSS 1.0, we can just return. If it is pre-RSS 1.0 we have to use the content-transformer to pull out RSS 1.0. We then return that.


SubscriptionProxyUpdateTask rework

Now that all subscriptions are in the database. We need to make sure that we update them from the DB instead of subscriptions.xml. The SubscriptionProxyUpdateTask needs to make sure that we have registered tasks for EACH subscription.

Right now we are using SAX to parse out subscriptions.xml. We need to use the OM layer to get the subscriptions and use them directly.


meta-update of the TEXTINPUT table

Right now there is nothing in this table. Reptile will need to support this in the meta-update. If it does NOT then we will not be able to provide search forms for Reptile channel views.


UpdateSubscription action.

Need to migrate this to use the DB.



TODO

  1. Make a UML sequence diagram of this when we are complete.
  2. Migrate this document to formal documentation when complete.
  3. FIXME: could we use this as a mechanism for spitting our RSS for any channel (even from the past)



Copyright © 2001-2003, OpenPrivacy.org