Edited by author 06-18-2001 04:33 PM
This is a spot for discussing the threading standard Steve Yost has proposed. An article in JOHO talked about it:
http://www.hyperorg.com/backissues/joho-jun17-01.html.

Dave, what about Radio Userland (http://radio.userland.com/)? All of the threads are in XML, XML-RSS is fully supported (and them some) -- it would really seem to fill the bill. Plus, you could set up your own feed, so there wouldn't be such a drought between tasty drips from your incomparable lips (!?>).

Stan

Edited by author 06-18-2001 07:54 PM
Doesn't Steve Yost cover this issue where he says (on his FAQ page):
"You can get something like threads -- branching a new topic from another -- if you use the "Start your own topic" link on your topic page, and select the "Link previous topic to new topic" checkbox. When you do that, a message gets created in the first discussion space that links to your new discussion, effectively branching off there. A link appears in your new discussion space back to the previous discussion, so you can easily jump between the 2 topics."
My first impression is this would improve on the usual threading where so often people ramble into a new direction but leave an older (now inappropriate) Subject heading on their messages: the dreaded "off-topic" malady.
Cheers, and thanks Dave for introducing QuickTopic in JOHO.
Bruce

RE: #2, Stan.

Which bill are we filling? Sorry, I have the attention span of a gerbil.

Stan, RSS (RDF Site Summary) is great for syndicating newsfeeds and similar sources like weblogs, and Radio Userland is great for gathering and publishing these feeds.

However, the message thread standard discussed here is about the following:
taking a thread of messages already posted among several people using a single app (like email or a bulletin board), and copying that entire thread to another compatible application or service.

Each thread consists of messages written by several people. We want to be able to save it and use it somewhere else, exchange it directly between two cooperating apps, or just archive it in a standard format.

Here's an example scenario: I'm in an IM session with someone, and the conversation ends up worth saving, and I'd like to involve other people too. So I save it (my IM client keeps a message log and allows me to save the thread in the standard format). I then create a Quick Topic forum pre-loaded with that conversation (QT supports the thread standard, allowing upload and download), and invite others. (Even better scenarios involve direct communication between IM and QT, or between email and a BBS, etc.)

In David's case, his web BBS went out of business (as many have), leaving him to copy and paste his messages; with a thread exchange standard, he could have saved them to disk in one step and uploaded them elsewhere.

The main questions in defining a thread exchange standard seems to be these:
What are the core set of attributes that make up a message thread? What are optional attributes that certain applications might want to support? For example, core attributes would probably include message author, message posting date-time, and message body. An optional attribute would be message title (note that RSS requires title, link, description, etc. because they're core to syndication).

Should the standard allow representing a hierarchy of threads? These occur often on threaded bulletin boards, newsgroups, and even in email (though they get pretty cumbersome in email).

I could really use the help of an expert in defining XML DTDs. I've created one before, but for a broad-based standard like this, an expert is called for.

Edited by author 06-19-2001 12:15 AM
David, the W3C Note that you cite in your article is about ways to use HTML to do the gawd-awful kinds of "threading" (really quoting) we all do in email when we reply and quote the previous message with '>' indentations or whatever. Here's the relevant excerpt:

Some applications of HTML Threading include (but are not limited to):
A mail UA may allow the user to specify and attach their own personal style to text that they author. (Note that this may be overridden, ignored, or even made invisible by the recipient.)
Formatting may be used to distinguish text written by various authors, or that originated in different messages. This formatting information may be provided by the sender of the message, or provided locally by the UA.

So that's an entirely different beast, and your proposal at the Kit Kat Club is still the world premiere.

Edited by author 06-19-2001 01:37 AM
I'm a little confused here: Hasn't USENET been threading messages (and cross-threading) for nearly 3 decades? years ago I built a prototype forum for a large telco and used NNTP as the underlying technology; the forum was really nothing more than the formatted display of multi-part mime messages (ClariNET was a contemporary example applying this to the news portal idea).

I figured people could use it like a blog (we didn't have that word then) or use a newsreader (which far too many still don't know how to launch let alone use) or they could even keep in the discussion via email/usenet bridging. Best of all worlds, I though, and yet, I couldn't interest any of my big clients ("No one uses USENET!") and in all the years since, I've only seen one other site use this method (a Linux news site).

what was so terribly wrong with using NNTP?

I do like the idea behind quicktopic, although I'd want an opensource implementation before I bet the farm on it (so I don't get caught out like a certain subscriber of CoolBoard that I know ;) ... and if I had the code, I'd be very interested to see how it might integrate with the RSS-based Peerkat to produce forums which have no central host.

RSS 1.0 has a proposed construct ("mod_threading") to describe the thread parent/child relationship structures. The key question (as Steve says below) is whether RSS has enough, or too many, other attributes.

My feeling is that, using Dublin Core, it certainly has enough expressivity; and that although the item title in RSS is a required element, a blank title is OK.

Threading spec: http://groups.yahoo.com/group/rss-dev/file.../mod_threading.html

Edited by author 06-19-2001 10:14 AM
Gary, let me be lazy and quote the NNTP RFC: "NNTP specifies a protocol for the distribution, inquiry, retrieval, and posting of news articles using a reliable stream-based transmission of news".

As we'd apply it here NNTP (network news transfer protocol) could be used as a protocol for transferring threads of messages between apps, but what we really want is a spec for statically representing a thread, either on disk or as a payload during a bulk transfer. NNTP deals with nice things like only sending new messages, only sending headers, etc. The related RFC 850 specifies (among other things) how threads are actually represented: using the 'References' field in the message header. 'References' is an optional field, BTW, and it's up to the app to represent a thread using it. Here's relevant text from the RFC:

The purpose of the References header is to allow articles to be grouped into conversations by the user interface program. This allows conversations within a newsgroup to
be kept together... User interfaces may not make use of this header, but all automatically generated followups should generate the References line for the benefit of systems that do use it...

All told, RFC 850 looks like a good model to work from, choosing items as applicable from its required and optional fields. I like this: "The primary consideration in choosing an article format is that it fit in with existing tools as well as possible." That specific goal applies here; it's just a slightly different set of tools.

[Update: RFC 855 is superseded by RFC 1036]

Seems that we need a really clear statement of the goals and non-goals. For example, do we want the NNTP-like capability of choosing a range of messages by date, so we could support updating a thread from one app to another? That's the domain of a protocol, which we'll inevitably come to. We should work on the format first (RFC 850 came before NNTP [RFC 977]). Right?

Good grounding in related history is important. So thanks for making me take a closer look at NNTP & family sooner rather than later, Gary.

Next I'll look at your reference, Hugh. Though I think RSS is solving a different problem, I may find that Dublin Core (a good name for a nerdy drink -- Guiness and hard cider?) is applicable, though RDF and its meta-specs have made my head hurt (much like that drink would...).

I thought the Dublin Core was a standard for documents with an eye towards their management, including making them findable via metadata. Our semi-pre-proposed threading standard is more about messages and, especially, about their threaded relationships. The closest I can see in the Dublin Core Metadata Element Set is "relation":

 ---
  Definition: A reference to a related resource.
  Comment: Recommended best practice is to reference
               the resource by means
               of a string or number conforming to a formal
               identification system.

 ---
http://dublincore.org/documents/dces/

While msgs would obviously need an identifier pointing to the thread in which they're contained, we also need (or so we're claiming) a standard way of expressing the thread itself. For example, a thread can have an owner, may be open or closed, may have an expiration date, etc.

So, while I'm all in favor of building on existing work, and while I've long admired the aims of the Dublin Core group, I'm not sure I see a strong connection. Unless I'm missing something and/or things have changed. (Usually one of the previous two conditions obtain. Sigh.)

Edited by author 06-19-2001 11:29 AM
Here's a try at a philosophical summary:
A conversation, consisting of a set of messages among several people on a particular topic, is potentially an asset that's worth preserving. We want to assure that any participant in a conversation can easily save the whole conversation in one piece in a standard format. We expect these benefits:

• The ability to archive a conversation for later reference.
• The ability to transfer the conversation as a whole from one tool to another. Reasons may be that another tool becomes more appropriate (e.g. for widening the audience), or that a particular tool (e.g. a web service) becomes unavailable.

Does that capture it, David?

An open question is how we define the continuity and bounds of a conversation. A conversation is explicit in IM (all the messages in a session) and in Quick Topic (all the messages in a topic designated by the URL).
It may also be clear in large-scale web-based discussion forums where one explicitly posts within a specific topic area.
It's less clear in email and news, because a conversation thread is actually pieced together and represented by the client application (the email client or newsreader), using the optional 'In-Reply-To' or 'References' header fields, or if they are unavailable, the 'Subject' field. But because they *do* manage to do this, they can presumably allow the user to select an entire thread and export it in a standard format.

A related question regards the branching of threads within a conversation. What do we want to capture -- only a single linear thread, or a set of message nodes in a tree structure? In email and news, a tree structure is inevitable -- more than one message will have the same messageID in its 'References' field (i.e. the same parent message). Supporting email is reason enough to support the full tree structure.
Here's a proposal: Applications that can't represent a tree should "do their best" to represent it^*. That would mean that a round-trip (e.g. email to a BBS and back) might not preserve the original thread structure. The one-way interoperability is more important than round-trip preservation.

David, this may mean that messages simply have parent message IDs, not a thread ID, though the entire conversation would of course be encapsulated in a single bounding structure. WDYT? The other attributes you mention: owner, open/closed state, expiration date, are useful -- I think they'd be Optional attributes.

^*Quick Topic, for example, would represent threads using pseudo-threading: creating new topics that link to the main topic -- see the FAQ (bottom of page).

Good job capturing it. I'd add one more benefit bullet: The ability to extract additional value from the conversation, e.g., making it searchable or integrating it into knowledge systems. [Yes, I used the dreaded phrase "knowledge systems." Now I have to wash my mouth with a cleansing sorbet.]

Put aside the architectural question for the moment (because I am supremely unqualified to comment on it). It seems to me essential for "marketing" purposes -- and for clarity of purpose -- that we think of this as a standard for threads, not messages. I would think that this means that there should be a separate entity called a thread that has certain properties, including an ID, a title, permissions, etc. It may well be that the elegant way to architect this is to devolve those properties to the messages and thus not require that there actually be a thread-in-itself (Kant is spinning in his grave), but the user will think of the overall object as a thread even if it's just a linked list of msgs.

Further, yes, I think it has to capture all the msg nodes in a tree structure. The standard needs to be as robust and useful as possible so no discussion application will be left behind; this will be a measure of its success. I suspect that there's some fiendishly way enabling a simple architecture to capture indefinite complexity (the RDF standard does this, I believe), but that's not my department (says Werner von Braun)...

I am now done pontificating on that about which I know naught.

Yes, definitely, it's a standard about threads -- perhaps we should say conversations, since the things of interest may contain several branching threads. Anyway, it's the ordered aggregation of the messages that has the value. Let's keep that clear when conveying the marketing message (please pass the sorbet).

Is a conversation a first-class thing-in-itself? Does a conversation have an ID? Here's a thought experiment. Let's say you pick up an email thread that started last week and export it to a knowledge base. Next week I pick up a branch of the same thread that's diverged to another topic, but which contains some of the messages that you picked up. I export it to the same knowledge base. Your *framing* of one conversation and my *framing* of another conversation each added value to the respective set of messages. We'll probably each attach different keywords to our conversations and file them differently. Our knowledge base may want to recognize somehow that there are messages in common between the threads, or it might not -- that doesn't seem to be primarily important.

So we can think of these overlapping message *frames* (the conversations) as different entities, though we may (or may not) want to recognize the messages they contain as overlapping.
I still don't know if a conversation needs a unique ID (as messages do) -- what would the ID uniquely identify? My act of framing it? I guess so, especially when I've attached descriptive metadata.

Hmm, given the above arbitrary framing, I don't think that open/closed, author, or expiration date are useful after all. What do you think? The thing is, some tools (like QT and IM) start with a concrete conversation boundary -- others don't.

IMO, if I clip a branch of a threaded discussion and stick it on a different board or even stick it onto another branch on the same board -- something that I don't think any board lets you do now (?) -- I'd maintain that we now have two distinct threads. Yes, it would be good to know that msg B is a copy of msg A and that msg A lived in Thread A. But that's a complication that's at the fringe of what we're trying to capture.

I want to come back to the marketing side of this. We're proposing a way of preserving those things called "threads." As a user, I may want to do some operations on those threads such as delete them, rename them, store them, mark them closed, etc. Whether the architecture maintains a sep. thing called a "thread" is way beyond my competence. Whatever works ... so long as the end user gets a presentation of a thread-in-itself, even if it's totally derived from the metadata of its constituent msgs.

Edited by author 06-25-2001 10:31 AM
RadioUserland has been mentioned here before. I just wanted to add that it not only supports RSS but also OPML, an XML format made to handle outlines. It might be a start since it's threaded by nature.

See http://radiodiscuss.userland.com/howToUseTheOutliner

I think there's a page dedicated to OPML but I can't access any Userland sites right now.

Update: Userland is back online, here's the link: http://www.opml.org/

Thanks to Steve for inviting me to take part in this discussion. I've done a lot of work building formats in XML, RDF and RSS and I'd love to help out with the format. My initial thoughts (which seem to coincide with some of the other comments in this thread) is to go with RSS 1.0. Here's an excerpt from my message to him:


The first thing that comes to mind is to use RSS 1.0[1]. It's a
widely accepted standard for syndication, supported by tons of
platforms and products which means that you'll be able to follow
discussions in the format with tools like Meerkat, Radio
UserLand, Headline Viewer, etc.[2]. It's also supported by
ForumZilla[3], a discussion system for Mozilla as well as built
into SlashCode (the software that powers Slashdot and other
similar sites). It'd also give you a lot of parser support from
tools like XML::RSS, and Orchard-RSS.

RSS already has a comprehensive content module[4] capable of
sending content in all sorts of different forms. There's also
been some work on a threading module, which, if you're
interested in going ahead in this direction, I can try to
collect them and put them together into a module draft.

[1] http://purl.org/rss/1.0/
[2] http://blogspace.com/rss/readers
[3] http://www.zapogee.com/forumzilla/
[4] http://purl.org/rss/1.0/modules/content/

Aaron, looks like there's just a few of us subscribed (I get to look behind the scenes and see who :-), including David, so let me just ask what you think of my statements in message 5.

I assume you mean the issues you raised about RSS requirements. You wrote:

note that RSS requires title, link, description, etc. because they're core to syndication

First, RSS doesn't require description. Second, it's easy to automatically generate a title based on other things (time, participants, or just plain "Untitled") as well as *some* sort of link, for those feeds for which this doesn't make sense. I think the benefits of using the RSS format outweigh these costs.

Let's go with that thought and see what RSS does for us, keeping our goals in mind: a format for saving an entire thread of messages from one of several kinds of discussion apps, with the ability to import it into another discussion app.

There are the runtime goals (what the standard accomplishes for users of the apps that support it), and the adoption goals (the standard is more useful as more apps adopt it).

RSS has a great adoption advantage in already existing and being supported by several interesting (albeit syndication-oriented) apps.

My hesitation is the runtime goal -- i.e. what RSS is intended for. Syndication is about periodically gathering summaries of the latest items from one publishing node and publishing them within another.

But people use RSS for other things (probably following the same reasoning we're using here: it's out there and it works), and that's one thing that caused the push to RSS 1.0, using RDF instead of a DTD, as Edd explains here: http://www.oreillynet.com/pub/a/network/20...zine/rss_intro.html.

Edd's article helps me in another area too: my big concern about crafting a DTD is how to allow for extensibility and how to carefully choose which items are optional. RDF allows that more flexibly it seems, at the cost of introducing a new meta-level of description. Aaron, were you a part of the decision making regarding the move to RDF? What were the considerations?

---

OK, now let's flip and ask from the other side: if we were to create a DTD, is it feasible to produce a DTD that's flexible enough? Can DTDs be extensible? That's where your XML expertise is needed, if you'll allow the thought experiment of a non-RSS solution for a moment.

You know, I always work best with an example. Aaron, can you (or Dan Brickley, who's joining us -- welcome, Dan!) do a stab at an RSS example of, let's say, part of this thread?

You'll of course make some assumptions in mapping, so don't treat it as a formal proposal. I'd mostly like to get a better idea of how you use RDF to be flexible.

If you want to do it, say so first so you don't both spend your time on it (unless you want to of course -- it'd be interesting to see the variations).