Steve Yost suggested:

Requirement: It seems that most threads are recognized as such *after* the first post. The original poster doesn't know whether his/her post will be picked up by anyone, and so if there's any extra work involved in designating a thread, it should fall first on the the first respondent to an original post. But how then does the thread get designated in the original poster's blog? The first respondent needs to notify the orignal poster, who adds the Needly link to their blog post.

The alternative is that all blog posts are automatically potentially thread initiators, owing to tools that write and read common formats. I guess this is where Bb is going with an RDF-based solution.

Steve

Edited by author 06-18-2002 10:29 PM
One thing I'm trying to wrap my mind around: the whole process seems really complex, with a lot of parts and requirements for intentional action for each post to be included. I think we could create very much the same functionality more simply and with less required interaction as follows: a blogger would subscribe to a service once. The service would keep an eye on the blog's regular RDF file. When it spots a new post, it would crawl the text of the post for links to Permalinks in another subscribed blog. If such exists, there's pretty much a parent-child relationship between the two messages. The service would add that to a tree of links. Would this indeed be easier (I admit to not being enough of a programmer to try it out) and would it miss any of the information that the original proposal would include?

Edited by author 06-19-2002 03:45 AM
Cross posted from Gary Turner's comments thread :

I've posted thoughts about an app vaguely like this a few times, which I'll link to, rather than take up space by reproducing here :
here, here, and here.

Not really having thought much about what was possible, though, what I was talking about is perhaps less achievable than the (at least at the beginning) manual opt-in solution that BB is talking about.


What I see as a potential problem (if I understand the proposal) is that someone has to know that they're starting a conversation thread, or know that they're continuing one, in order for their post to be included in the thread. This might lead to some chaos.

Does that make sense?

(I think in essence what I'm talking about as potential 'chaos' (too strong a word) is also what Joseph is talking about, below)

Edited by author 06-19-2002 08:00 AM
Good to see we're all moving along the same path. I mentioned this problem (let's call it "Who's On First?") to Bb and she didn't feel it was an overwhelming problem.

Perhaps "all blog posts are automatically potentially thread initiators" (as Steve Yost suggested). My initial response was to dismiss this since I know that not all of my posts are likely to provoke cross-threaded discussion. Then I thought about the posts that I'd regarded as unimportant which actually attracted lots of comments and posts in response. So maybe that is the best mechanism.

Alternatively, blogger B can nominate a post by blogger A as a "thread initiating post". Post A then initiates the thread and post B continues it. If blogger C also tries to nominate post A or post B, the system arbitrates and adds post C to the thread.

I don't see that there's any conflict between this manual method of alerting the system to threads and the automated system that Joseph suggests.

I guess what I'm trying to say is that Bb kind of anticipated this issue in her original design.

What makes me feel optimistic, though, is that we've all locked onto the same potential difficulty -- and that's exactly what she was hoping for (that we'd try to anticipate problems).

I think this discussion also indicates that a sign of Needly's success would be that DayPop or the Mighty Google starts to build blogthreads automatically out of the blogs it spiders. When I search at Google for "forgiveness" (as opposed to searching at Google for forgiveness, which would be rather pathetic), I'd love Google not only to notice AKMA's initiating blog entry on the topic but for it also to let me click on a blogthread glyph and see the entire Needly web.

Should there be anything in the Needly standard to facilitate this? Or is it an irrelevancy?

Edited by author 06-19-2002 01:40 PM
Another related thought that strikes me: perhaps we could get our blogs to include links for each message that would link back to the service and check if it is part of a logged blogthread, bringing up the thread(s) if it is or an appropriate message if it isn't. In Personal Weblog, the blogware that I use, it could be done with one line of code, which would add something like BlogThread

(actually, the @MYURL@ disguises a faintly more complex URL, but you get the idea).

(Grrr, even though I used ampersand-l-t-semicolon and ampersand-g-t-semicolon to actually show the code, QuickTopic pseudointelligently converted them to the corresponding arrows, turning it into an actual link. You'll have to hover your mouse over the link or view source to see it.)

(And I see that QuickTopic sent out email to the folk who subscribe to this convo before I had a change to see the resulting message and edit it, and won't send out a corrected version. If whoever designed this is reading this thread: could we please have a Preview Message feature?)

Another thought: Bb's original message described the displayed tree as consisting of just the posters' names. I think having the posts' titles on the tree would be important too, since often what people comment on isn't the main thrust (and hence title) of the original post. It might also help spot topic forking.

I wish I were techy enough to join in the discussion of "how,"--I'm not. But I'm way excited about the prospects of when, where, why, and even who.

I second what David says below, and Joseph Zitt too. I don't know how possible it would be, but it would be cool to be reading david w.'s blog, for example, and then right click on a post and see all the related, threaded posts with links right there. If all of us did it, you could go hopping madly about, which, though maybe not the most organized way, is always my favorite thing to do (the hopping madly).

Maybe we're responsible for "adding" related posts to our own threads with a little "add post" button in that pop-up. that would let us link posts that don't necessarily speak to our post, but maybe pissed us off or engaged us enough in the first place enough to post. Then of course you'd need a little "delete related post" button in case you added the wrong one by mistake. IT almost becomes like a mini blogroll inside a post.

Whatever I have to do to enable my blog(s) eventually, I'll do. Someone just has to tell me the 'how'. And I have no idea if I know what I'm talking about here. Just talking.

luvvvv,
jeneane

Hi.

See Andrea James' comment

http://rateyourmusic.com/yaccs/commentsn?b...id=85177710#1342072

"Hmmm... I'm not sure how the deliberateness of the threading functionality leads to more chaos"

I somehow agree, at least in the worst-case scenario,
this could degenerate in a kind of super /., with posts
distributed across blogs.
The problem with /. being the sometimes
huge number of replies (with very variable
relevance)

Maybe the time-cost to attach a post to a thread,
or simply common sense, politeness or the different
blog culture will avoid this kind of problem.

Otherwise a kind of mechanisms to sort replies
by "relevance" would be needed like in /. .
(making things even more heavy-weight)

Or the service should be distributed, one service
somehow per local blog neighbourhood - something
Shelley Powers suggested too - but then the mechnism
will not help crossing the blog neibourhoods' chasms.

Just my (maybe not all new) 2 cts.

regards, Samuele Pedroni, major blurker

The problem I have with this is the extra work needed by all parties to keep it going. The beauty of Google is that it derives the meta
connections from existing content without needing all content to be changed, which is how the made meta keyword tags obsolete.

Here's a counter proposal:

We need a way to detect the span of a single blog post, and identify an incoming permalink to it. Then we can make a spider that crawls blogs and shows the threads implicitly. We can create the RSS feeds for the threads as as Shelley describes.

Blogthreads are often rooted in an external (non-blog) piece - this is the kind of linkage DayPop shows - the feed could thus point to these root items too, showing the blogs that discuss them, but as they won't contain the blog post delimiters, the spider will not follow links from these source pieces.

One could do this purely via RSS and follow links within RSS posts only, but the variation in RSS usage (some include titles, some the first para, some the whole post) would make this tricky.



On Wednesday, June 19, 2002, at 04:59 AM, QT - Jonathon Delacour wrote:
>
----------------------------------------------
weblog: http://epeus.blogspot.com
< replied-to message removed by QT >

Edited by author 06-20-2002 10:00 AM
I wonder if this could be implemented simply:

In your blog entry, use <bt name="Forgiveness"><a href="http://www.seabury.edu/faculty/akma/2002_0...garch.html#77084283">AKMA on forgiveness</a></bt>

A blogthread-savvy spider looks for the <bt name=...> tags, and examines the hrefs between them. It needs to be smart enough to determine the blog owner based on the href (doable given a list of blogs). It rebuilds threads bottom-up on each spidering run.

Half-baked. Please poke holes. Hmm, one problem: readers can't see the <bt> stuff. Maybe just use visible markup?

(Noted your QuickTopic problems in /m6, Joseph. I see what you mean about the lt's and gt's. Will try to fix soon.) [Later: it's now fixed.]

I think the Permalink could be identified by a tag that would be automatically put into the code by the blog software, something like 'permalink="yes"'.

This gets me to wondering: how many different blogging softwares are there, and are their developers approachable?
I suspect that maybe seven programs account for over 90% of blogs, but I've never seen a survey.

(Steve: thanks for the note (/m10) on the problems. I didn't mean to jump down yer throat, and realize that I sounded really grumpy in that message.)

timestamping is a critical thing here I think, ordering the thread by time posted or something. I guess Bb is on the case in terms of the logical threading issues and the solutions or not therein. From a user perspective if I post something which is my take on post number one of a thread elsewhere or post number 20 I guess all I want to do is to link or refer to the general thread header which needs therefore to be contained in the link reference e.g. www.garyturner.net/archive_0602#48384373#thread_id, therefore there needs to be some embedding of that id into the posting templates so that it gets included when posting everything.

Bb's app then needs to scour sites for unique thread id's, sort them by time posted (UTC) and collate them?

I have no idea if what I'm talking about here make any sense at all...? If i had a brain I'd be dangerous.

On Wednesday, June 19, 2002, at 03:21 PM, QT - Gary Turner wrote: >
> timestamping is a critical thing here I think, ordering the
> thread by time posted or something. I guess Bb is on the case in
> terms of the logical threading issues and the solutions or not
> therein. From a user perspective if I post something which is my
> take on post number one of a thread elsewhere or post number 20
> I guess all I want to do is to link or refer to the general
> thread header which needs therefore to be contained in the link
> reference e.g.
> www.garyturner.net/archive_0602#48384373#thread_id, therefore
> there needs to be some embedding of that id into the posting
> templates so that it gets included when posting everything.
>
> Bb's app then needs to scour sites for unique thread id's, sort
> them by time posted (UTC) and collate them?

This is a gaping hole in RSS - there is no spec for machine-readable timestamps, and none for per-item timestamps either.
These would not be hard to add for any blogtool; just getting agreement on a format and name is all that is needed.

I suspect that ordering just by timestamp and thread ID may be insufficient, since people read blogs with different frequency and in different orders. While it's probable that a person who posted a given message did not yet read a message that was posted after it, we can't assume that the person had read the earlier message yet (or at all). The only evidence we have are the actual links.

And even this can get weird when you consider messages edited later and repeated references within a single message, as in David and AKMA's pas de deux (sp?) this morning.

Edited by author 03-19-2003 01:14 PM
If it's thread-oriented, and threads can be inferred by following links bottom up, are timestamps absolutely necessary? Yes, they're useful if two replies are at the same hierarchical level, as in Jeneane and David's reply to AKMA below:

AKMA
 Jeneane
 David
  AKMA

But if timestamps are a monkey wrench, maybe we can live without them. BTW, RDF certainly supports timestamps through Dublin Core. QuickTopic's RDF (just add ".rss" to the URL for this topic to see it) uses it. (BTW, this is a preliminary instance of ThreadsML, which is RSS-compatible.)

No replies yet to my simple-minded suggestion (/m10). Let me elaborate and modify it:
By convention, when I respond to someone's blog post in my blog, I already link to their permalink. I propose we add a convention of adding the visible text "[bt]" just after the link. A blog-specific spider notes this and adds the just-preceding link to its data tree of thread nodes, and attaches a node for the page it's currently spidering.

But hmm, many blog tools create several versions of a post: e.g. the most-recent page, a monthly archive page, and maybe a single page corresponding to the permalink. For the spider, the isolated post in a permalink page is preferred, and the link before the [bt] makes that nicely explicit. But how can the spider intelligently replace non-permalink nodes that it added earlier? (This is probably clear as mud -- let me know if I should explain further.)

Anyway, taking it a little further: instead of just plain-text [bt], the [bt] is a general link to the ThreadNeedle service. Then when a blog reader clicks the link, the service notes the referrer, matches it with a known blog, and shows the most recently updated blogthreads that include that blog.

Is this too simple? Too much of a hack?

Wow. The rss in here is way more complex than anything Blogger or Radio generate.

http://www.quicktopic.com/14/H/P96CAYje2yc/p19.11.rss

Could one just adopt the <dc:date> without all the rest? Anyway, point taken about dates being redundant for threading purposes; they are useful to know when the posts were made though.

Let me back up a wee bit more and check we have the same goal here, as it seems I'm thinking about this in a different way. What you are talking about is a new markup convention bloggers can adopt to indicate when a post is part of a thread.

What I'm talking about is a way for the existing implicit blog threads to be discovered and enumerated by a spider. If the concern is that it would over-identify threads, you coud provide a way to indicate that a blogpost isn't part of a thread - out out rather than opt-in.

Edited by author 06-20-2002 12:55 AM
> way more complex...

Yes, the RSS here is a stab at ThreadsML, actually a superset of RSS 1.0. More here on that. David helped kick this off. Regarding just using dc:date, you can certainly mix and match modules when using RDF, which is what RSS 1.0 is based on. Consumers of the RDF can just ignore what they don't want to deal with. (That's why RSS readers like AmphetaDesk can read QuickTopic's RSS.) But of course we should specify exactly what will be used if there's to be a BlogThread RSS module.

> Let me back up...
I think we agree: I'm suggesting a markup convention bloggers use so spiders can better discover and structure threads using that notation. Further, readers can easily get to the thread listing from any post in the thread. The goal is implied, but should be restated: blog creators can consciously indicate blogthreads, so their readers can easily see (and follow through links) the structure of the thread.

Oh, but wait, I see (sorry for the late editing). You're suggesting, in saying "opt out vs. opt in" that threads should be implicitly discoverable. That's very reasonble; I'm just looking for a cheap simple way to make it easy to implement (my usual approach :-). I kinda like the referrer thing that results too.

Edited by author 06-20-2002 01:21 AM
By convention, when I respond to someone's blog post in my blog, I already link to their permalink. I propose we add a convention of adding the visible text "[bt]" just after the link. A blog-specific spider notes this and adds the just-preceding link to its data tree of thread nodes, and attaches a node for the page it's currently spidering.


But what I'm thinking of doesn't even need that: all the spider would have to do would be to notice a link from one blog that it knows to another (ignoring, perhaps, links to the blog's main page). As I see it, if you're linking to a message in another blog, there's a thread happening. The [bt] might be redundant.

(This doesn't however, address the issue of the linkback to the ThreadNeedleCatcherThingie system's page about the thread, but I suspect that nailing down a way to do that and getting the blog softwares to include that might be pretty simple, especially seeing how quickly the RSS automatic discovery thing took off quite recently.)

Edited by author 06-20-2002 10:04 AM
Side note: Joseph, I fixed QT so it preserves explicit &lt; and &gt; in hrefs. I edited /m10 to test it.

Edited by author 06-20-2002 10:33 AM
Many blog tools produce static HTML, i.e. the blog pages aren't served up dynamically. So I guess they'd need to support an explicit user command that updates the entire blog for blogthread links (probably talking to the blogthread server).

I still like the referrer alternative though. For non-techies: when you follow a link with your browser, your browser tells the web server at the destination where you came from. Here's an example: [bt]. The destination page, if it's dynamic, can then behave differently depending on where you came from. It's not 100% reliable because some ad-blockers or firewalls strip this information out.

We should get Burningbird here for some feedback. What do you say, Bb?

I just want to intervene for a second to say that I sure appreciate your vigorous interest and support for this discussion, and I sure wish I understood more of the technical matters you’re discussing.

May I propose a metaproblem that might not easily be solved with code? We may run into problems with determining "what belongs to a thread." If Joe Crank wants to say something about forgiveness, do his rants enter the network on the same standing as Steve Yost's thoughtful observations? If he writes dozens of little bloggettes to crowd out the rest, is there a way to filter him out?

And what happens as the topic shifts--our network of threads may start out talking about forgiveness, and modulate to a different theme altogether, attracting a different bunch of responses.

I sense that we may need some kind of refereeing function. That could belong to the thread-starter, or perhaps distributed in some way among participants, or some Central Scrutinizer could determine what goes where. But I wonder whether protocols and code can solve this problem.

WRT /m21: The standard way of representing the blogthread should allow someone to write an app that lets someone edit the blogthread and generate a new one so that, e.g., AKMA on his site can write "To continue the blogthread on forgivenesss..." with a link to a blogthread that strips Joe Crank's comments out.

Also, someone should be able to write an app that displays the blogthread so that the more links to a site, the more prominent it becomes, a la Google.

Hmmm. This is sounding more and more like threadsML...

Edited by author 06-20-2002 06:26 PM
Perhaps Shelley has already figured out the tech issues around tracking and collating threads, we'll find out soon, so maybe we should aim to define the ideal features from a user's perspective, presuming no awareness of what the technological limitations might be?

WRT: /m23:

This is the user experience I would like: I read a post in a blog. I'm curious about its context, so I go to right near the current Permalink marker, where there's a link labeled "BlogThread" (or something like that). I click on it and am taken to a page on the Thread server which shows me a formatted list of links to the messages in that thread.

I decide to further the thread on my blog. I post a message, with a link, as would be done now, to a post on another blog.

The Thread server spider frequently scans the RSS feeds for the blogs that have registered with it. Seeing a new post on the blog, it swoops in and looks at the post. If it sees a link to another registered blog, it looks to see if that post is part of a known thread. If so, it adds the current message to that thread; if not, it has spotted a new thread and creates a new thread listing with links to those messages.

Possible problems: where to put the post within the thread tree if (as many posts do) it references multiple posts. And what if it references posts from previously distinct threads? (Will we end up discovering that they're all part of some Ur-thread?) And will there be human (or mechanized) intervention in case of thread spamming?

I'm on the road with a very limited connection - very glad to see this discussion.

Gary, I do have a way of tracking and collating the threads, but always willing to hear better ideas. Also, by embedding RDF directly in the weblog page, others can also grab the info, do their own collation.

AKMA, good point on screening, but the whole concept of this is openness, and gathering in weblogs at the edge - the people who don't normally get linked to or participate in the discussion. We may have problems with someone doing many blogettes, but I'm not sure - there's a little effort to this that hopefully will chase away the jerks.

Joseph - that's exactly how Needley works - click a link and you go to blogthread page, click another link to get code to add to your post to add yourself to the thread.

Needley actually works directly off of the interaction - just in time recording. The RDF is there for other applications to be able to scrap pages for blogthreads.

This is not - repeat - not going to be anything to do with RSS. RSS is an aggregation/subscription service. Needley is something different. The whole purpose of RDF is you can create different vocabularies for different purposes and uses, and use the same technology with each.

Needley's vocabularly has nothing to do with RSS.

As for multiple posts, embed the Needley code for each, and your post gets hooked to both with an asterisk to signal that it's replying to many posts. Or something along those lines.

David - not prominance - Needley's one any only primary purpose is to destroy buzz, popularly blogsnobbery, the whole thing.

Now, with the blogthread page, each time a new reply is added to a post, it gets output the to the updated discussion page - popular discussions will show up more frequently. But anything to do with "ranking" and I'll be death on it. We have too much of this already.

Did this catch me up?

P.S. Still on road, connectivity will be sketchy

How rude! I ask people for input and then blow in, slamming ideas right and left. Blame it on a long day.

Steve, I see what you're talking about with the link, and this does have the least moving parts. The main reason I wanted to embed RDF directly into the post is so that other agents can consume it. I was hoping to eventually not have the only ThreadNeedle application.

Issues of protocol - what to do with abusers, that's tough. So is issue of who "owns" the thread, and can you register another person's posting in order to start a dialog on it.

As for posting number - where you show in the list, since the RDF is generated by an application, it would be first come first served, don't you think?

David, Connecting to Needley would be simple for any app, Google, Daypop, etc. Since the bots from these services are in the general vacinity anyway, no reason they can't consume the RDF. We can even them a service and/or API to use.

One thing about thread ownership is people can register their weblogs - and adding a thread to a posting connected to a specific weblog generates an email to owner. Owner can also edit thread (but not remove - or would this unethical?_)

Edited by author 06-21-2002 12:18 AM
Bb in /m26:

Joseph - that's exactly how Needley works - click a link and you go to blogthread page, click another link to get code to add to your post to add yourself to the thread.

If you get a chance (I understand the road hassle thing), could you show us what the code to be added would look like? It might help us visualize what you're planning.

(I should probably go brush up on the whole RDF thing -- I realize now that I've been confusing it and RSS.)

Shelley - My point in response to AKMA wasn't that ThreadNeedle.com ought to show rankings but that the Needly standard ought to allow other apps to process ThreadNeedle's blogthreads to do things like: allow pruning and allow ranking.

That's the general point. But I think AKMA's point about pruning deserves further thought and discussion. Insofar as ThreadNeedle.com serves as the central repository for blogthreads, then I agree with Shelley that we want it to be as inclusive as possible, just as we want Google not to trim the idiots. But AKMA points to a different need. As it stands, if I want to refer people to a blogthread, I manually collect the URLs and publish them in a list. And I naturally exert editorial control: I trim the idiots (unless I aim for complete comprehensiveness); I think I'm doing my readers a service. As a blogster in the midst of a blogthread, I'd rather be able to refer people to an edited list than to the comprehensive list. The latter is more like saying "Click here to do a search on Google." There's value in that, but usually there will be more value to me in saying "Click here to see the list of the blog entries that I'm currently actually responding to."

So, I'm hoping that ThreadNeedle.com will be the comprehensive organizer of blogthreads but that the blogthreads will be nicely wrapped RDF objects that another app can extract, manipulate (e.g., trim, rank, reorder, re-display, index, etc.) and save somewhere. (The RDF is embedded in the msg, so I don't need to worry about a host for the blogthread object, right?)

Shelley's comment in /m26 that she hopes not to have the only ThreadNeedle app fills me with warmth like a bottle of imported beer. Mmmmm.

David's picking up on my point with regard to cranks; not that ThreadNeedle should automatically exclude losers (we still want to read what David has to say) but that somewhere, somehow, I ought to be able to decide, "I've had enough of Steve Yost and don't want to read any more from him" and not be bothered if he wants to post his buttocks off on a topic.

With regard to sequence, I wish it were possible to do something that conveyed the particular posts to which a given entry linked by a vector or something. Do you see? That way, if I posted late an entry that really only answered what Shelley said two days ago, the time of its posting is less pertinent than its place in the chain. I'm thinking this might require a graphic interface (for simplicity's sake--assuming I haven't already crossed that frontier long ago--one might depict links to other posts in green, and links from other posts in, what?, orange (some colors already being taken by convention for hyperlinks)?

The result would look like a sort of unhinged DNA molecule--but it might most effectively convey how a thread was going.

Slightly OT: but for one image of what a graphic link map might be like, check out the whacky search engine visualizer at http://www.kartoo.com/

I think all it would take to collect the data would be for the data to show what links to what. While I'm guessing Bb's looking for something simpler, her admirable openness in design suggests that if that data is collected, someone else might write a visualizer for it (much as kartoo visualizes results from other search engines).

--
http://www.josephzitt.com/blog/

Joseph, I was actually thinking of Kartoo--except I wish the connections were more intelligible. The Kartoo connections may be significant, but I couldn't make them out.

(Now someone's going to point out that the interfact had a big sign in capital letters saying how to interpret the linking-lines. . . . .)

Joseph, you can see my test weblog at needle.burningbird.net. View source and look in the post - you'll see RDF that I use for my Post-Content management system. Thread the Needle's RDF will be much simpler. Regardless, I used Jena to pull the RDF out and process it from the post. And I've used a Perl-based RDF API to also process it.

As you can see, the RDF is not visible in the page - nor would it be visible to audio browsers for the visually impaired. Adding it is nothing more than copying and pasting the generated RDF to the posting - doesn't require any external interface so ANY weblog tool can participate. Hopefully, though the tools will generate this themselves, and I'll provide API's if any of them want it.

As for consuming the RDF - no different than RSS except that we're not adding yet another file.

When this is finished, the copy and pasted material will also have a link that is visible to ThreadNeedle; clicking this link opens the discussion page. A second link opens a form for adding addition information, which is then used to generate RDF to add to a posting that's a reply to one or more other postings.

Hope this helps a bit - still in process.

AKMA, David - whole reason for embedding the RDF in the page is to allow any app to pull this information out, process it however they wish. ThreadNeedle really doesn't need it because I have the thread information when the person registers a new dialog or a reply.

ThreadNeedle is starting the process by generating the RDF for people to use. However, the vocabularly is quite simple and anyone can use any one of the numerous RDF API's to generate the RDF. And since each posting and reply is "identified" by the unique URL, we don't even have to worry about a shared identifier system. All of the blogthread tools can process the same RDF.

What is essential is that the tools agree where the RDF is to be found. By embedding it directly in the posts, rather than cluttering up RSS files (which are for aggregation, not discussion threading - not the same thing), or creating separate files, the functionality is open to all webloggers, regardless of weblogging tool.

Based on all of this, you can see that a person can go to one blogthread service and say "I own this thread - exclude David and Steve, but Joseph's okay". There is nothing in the technology to stop this. Same as there's nothing in the technology that stops Dave Winer from filtering weblogs.com - which he does.

Now, think about this very very carefully. And think about some of the interations you've had with webloggers in the last few months - and think about the danger of filtering. If this is what you want, you can have that - but, really sorry, I won't build this into the original ThreadNeedle.

I realize that I'm now opening the door on this one and that there are people all over who are coding a restricted, gadget-based version that allows people to rank and filter.

Finally - AKMA as for visual cues, I was going to indent, and perhaps use icons (graphics with associated alt tags) that differ between levels. Using colors is a problem with the visually impaired.

Bb - Looking at the RDF at http://needle.burningbird.net/ I start to get an idea of what you're going for. I don't quite understand how things would get hooked up with threads, but I trust this will become clear. And I'm just dangerous enough with PHP to say that, once this is nailed down, I'll try to implement it within Personal Weblog. I'd not the developer of it, but Mark has graciously accepted my patches and suggestions.

(And I'm amused by the /m32 reference to the RDF not being "visible to audio browsers". I'm also often thankful for the stuff that we can't smell on TV. *grin*)

Hey Joseph - I do need to finish the RDF vocabularly for this application. I should have that out for review/comments/modifications next week.

So far none of the discussion points reflects on the RDF vocabulary as much as it reflects on usage of the data, so I feel pretty comfortable with posting a draft.

As for filtering - perhaps what ThreadNeedle needs is a dynamic nature, similiar to database views. With views, you can merge data from multiple database tables, name this merged view, and then access the data from the view as if it was the regular thing.

So you could create a 'view' of a specific dialog and post a link to that - the raw data would still be accessible. And people can read the raw data view if they want.

What think of this concept?

(P.S. I can see I'm going to need help on the user interface - I'm lousy at UI)

And Joseph, I bet there's been times when you've want to taste something on TV...

Shelley said : "perhaps what ThreadNeedle needs is a dynamic nature, similiar to database views"

The ability to dynamically pivot your view of the thread in question, whether it be by personblog/datetime/shoesize or whatever other metadata is stored about the comment in question when it is linked into the conversation tree....

(I have no idea what I'm talking about, because I know nothing about RDF - someone want to post a link to a reasonably concise Idiot's Guide to RDF here?)

...would be a mighty useful thing, I would think.

Edited by author 06-22-2002 08:09 AM
wrt /m33: Shelley, that's exactly what I was asking for. I *don't* want ThreadNeedle to filter. I *do* want the ThreadNeedle standard and the objects it packages to be filterable/manipulable by other apps that may emerge. (Whether filtering is useful is something we can let the market decide.) So: Great! That's why RDF is the right choice.

wrt /m35: Yes! Just in case the basis of my enthusiasm for ThreadNeedle isn't clear, let me put it like this (and tell me if I have this wrong): TN will do three things for us bloggers, each important.

1) It creates a standard format for packaging up blog entries into threads. It will be the de facto RDF-based standard for blogthreads.

2) TN is an application that will create those packages. Blogthreads now will be objects we can point to. And applications will emerge that do interesting things with those blogthreads.

3) TN is the first of the applications that does some interesting things with bloghtreads. Although its list of features isn't yet solidified, TN.com will enable us to find blogthreads and display them. It may also let us have personalized displays them, as per /m35.

If I'm wrong, let me know...

Edited by author 06-22-2002 09:03 AM
David Weinberger wrote: "I *don't* want ThreadNeedle to filter. I *do* want the ThreadNeedle standard and the objects it packages to be filterable/manipulable by other apps that may emerge."

I'd like to put this a little differently. I want ThreadNeedle to allow filtering out of the box because I don't want to have to rely on (or wait for) another app to implement filtering.

My news client, Gravity, has a Bozo Bin. If Joe Crank gets on my nerves, I add him to the Bin and I never see a post from him again. Just like in Stalinist Russia -- one day the guy is in the official photograph, the next day it's as though he never existed.

I'd like ThreadNeedle to offer the same capability. The democratic intent of the application would be preserved because no-one's posts would be excluded but my democratic right not to have to put up with cranks would also be protected.

I join the chorus of bloggers who want the standard to be lean and inclusive, but to allow for an implementation that's manipulable.

You're quite right that exlcuding voices has a significant cost. I'm still thinking of circumstances in which I'd just as soon not hear from/be aware of some bloggers' interventions. (I just rejuggled my bookmarks folders to shift some blogs closer to the top, and others to the bottom, to effect a similar end on my end.)

I wasn't thinking about colors; that's a good point. But perhaps there's a similar way to indicate the vectors of reference visually, that users who have sight limitations could also employ (I don't know well how the special-use browsers finesse colors, graphics, and so on). The complications of reference in a blogthread seem more convoluted than a simple indented outline would allow. Perhaps one of us (I'm not volunteering) should look over the forgiveness thread to try to figure out a way to situate later posts that refer to more than one previous posting. I still suspect that some kind of graphical/pseudospatial schema might be desirable.

First a quick technical aside: I provided a more detailed reason about why I'm not going the RSS route with ThreadNeedle at my weblog. In addition, I'm writing a book on RDF and if any of you want to read the chapters that describe RDF, I'll provide draft copies. Note that these are unedited, reviewer documents, and that reviewer comments have not been integrated into the documents.

David, we're in synch and agreement in understanding regarding the core functionality ThreadNeedle will provide per your message /m37. We've baked the cake and are now working on issues of filling and frosting and how to cut.

Let's chat about data for a second - what kind of data should I record? We know that ThreadNeedle needs to record points of a dialog and that we need to allow replies being linked to more than one point (weblog posting). I was also going to record the following information:

Category (such as Metablogging or Politics or Religion, etc)
Keywords (user supplied terms to be used for searches)
Date added (date thread added)
Date updated (date thread updated)
Region (person's locale)

I can't think of any other information to record, and even region isn't necessary (thought it would be an interesting extension, but can pull).

What do you all want to know about an individual thread when you view a dialog page? What do you want to know about the dialog itself?

Jonathon, per /m38, you're right - ThreadNeedle needs a minimum functionality out of the box to prove the concept and provide a viable product. This will generate support for the RDF-based ThreadNeedle vocabularly and encourage its propagation (as we've seen with RSS). I think we have the core aspects down - time to have _real_ fun and work on the filling and frosting for ThreadNeedle out of the box.

AKMA, per /m39 - so where does my weblog fit in your little bookmark list, hmmm?

It could be very interesting to be able to support something dynamic in ThreadNeedle's initial application, if for no other reason than that I would like to see what people do with it. Webloggers have been passive receivers of technology - what would they do with technology where they have more direct input. How would they carve up the dialog? Could be interesting exploration of group dynamics.

Sure transcends the hell out of Capital J journalism, doesn't it?

Also, I'm going to need help, suggestions on user interface. My strengths are in back-end development. Unfortunately, we can't be too sophisticated as this could greatly slow up the initial release of ThreadNeedles. What do we need as a minimum, and how do we present it?

The danger here is that we end up reinventing Usenet once more. Heavily used threaded discussion become tiresome rapidly - following a non-linear narrative takes a lot of effort from the reader. This is why Slashdot had to introduce some kind of comment filtering process, and why most of us stopped using Usenet in favour of mailing lists and places ike QuickTopic. If ThreadNeedle takes off, someone is going to be its Cantor and Seigel.

The success of QuickTopic and gang blogs like the Small Pieces blog or GonzoEngaged is precisely that they impose a linear form on discussions, and require a certain etiquette in posting.

Weblogs posts already have a good way of referring to earlier parts of the discussion through hyperlinks; the problem that ThreadNeedle is needed for is to find the comments that point to the post being read. Thus TN is basically a way of 'fixing' the asymmetry of links. Care needs to be taken here, because this asymmetry is what makes Google work

One way I have seen on some blogs is by using referrer logs to derive a list of ways thet people got to the content. If TN can track the flow of viewers through the links it reveals, dn reflect this by thickening or reordering the connections, the problem of spamblogging can be reduced.

Alternatively, one could adopt an individual-focused variant of the Google trust metric, whereby bloggers you link to a lot rank more highly than those you link to sparsely - you could even use blogrolls to indicate this. The question is where and how one draws the boundaries around the dicussion participants.

Kevin, per /m42 - ThreadNeedle won't work off of pulling in links. We already have applications that provide this such as daypop and google and blogdex. ThreadNeedle is a way of tracking complex weblog dialogs, with postings across numerous weblogs. In fact, it's only purpose is to track multi-threaded conversations such as these.

As for etiquette, I guess the same level of etiquette that will apply to ThreadNeedle applies to weblogging in general.

What's going for us is that for the most part, the people interested in this type of technology also spend time - usually - on their weblogs. I don't think we'll see a lot of quick posters, particularly if ThreadNeedle isn't contributing to their ranks, daypop rating and so on. If anything, ThreadNeedle will get the buzz.

In fact, popular discussions will automatically show up in Daypop and blogdex because there will be a lot of links to the discussion thread added to weblogs.

I wonder is Daypop and Blogdex will begin to see ThreadNeedle as being somewhat viral in nature.

Edited by author 06-25-2002 10:45 AM
Kevin said : "Heavily used threaded discussion become tiresome rapidly - following a non-linear narrative takes a lot of effort from the reader"

I would add here that there has a been a great deal of discussion about this over the years at Metafilter : that one of the reasons that MeFi works as well as it does (or used to, some would say, for unrelated reasons), is the fact that all discussion is flat and unthreaded. This is contrasted to the various thread design options at Kuro5hin and Plastic, which, although they have been finetuned to better useability in recent times (particularly the innovative 'dynamic' threadview at kur05hin) are still harder to mentally parse, IMHO.

This is a UI concern more than anything else, perhaps, but I feel very strongly about UI design (my pedestrian blogtemplates notwithstanding), and would also like to make sure that this (as BB requests) is carefully considered...

Is a thread the best way to represent the data? Perhaps it's the best, default way. What would be optimal, of course, is that the data from Needly (The Hot Needle of Inquiry!) could be sucked up and rendered by any UI-level component someone chooses to write. Me, I'd like to see it represented in an supercool engine like this one!

Oh, and I want a pony, too.

Edited by author 06-24-2002 09:45 AM
Data standard vs application

Agreeing with AKMA, I'll reinforce the point that we should clearly delineate the data standard from the thread-serving site. The data standard provides the potential for features implemented by the site. As Shelly pointed out in /m26, "The main reason I wanted to embed RDF directly into the post is so that other agents can consume it." Because the RDF includes more than just a link (/m40), it enables more features for the thread-serving site.

The standard probably doesn't need to allow for all the interesting uses. For example, an interestig application might combine thread presentation with Google or Daypop ranking. To the extent there are APIs for these services, ever-cooler web apps can be put together (of course there's always just hacking GETs and POSTs and parsing results). BTW, will Needly have an API?

As I understand it, the data standard is manifested as chunks of RDF embedded in weblog entries, one for every post that participates in a thread. Is there also a data standard for the aggregated thread that's gathered by the thread-serving site? Presumably it's either a simple ordering and bounding of the gathered RDF, or a superset of this.

Natural filtering, user walkthrough

I think that if it takes a little extra work to add a ThreadNeedle link to a blog, there will tend to be less noise than in the average threaded discussion. That's a positive byproduct; extra work is not a goal :-). Shelly, can you go over the actual steps for a blog author in detail?

Edited by author 06-24-2002 09:43 AM
Threaded vs. linear, Google rankings

It's been recognized among forum designers that threaded presentation (with only subject lines showing) works better for long-running, infrequently updated discussions that tend to meander, while linear, full-message presentation like QuickTopic's works best for shorter-lived, dyanmic conversations. Most conversations don't fit neatly into either of these categories, and each presentation has its benefits and drawbacks across the board. (I'd like your reactions to QT's features in the present usage, but that would only be self-serving :-)

Seems to me, though, that Needly's (the application's) purpose is explicitly to present the structure of conversation, presumably as a network of links (and yeah, Stavros, Thinkmap or The Brain[warning, long load time] would be very cool).

Blogthreads are an add-on to the hyperlinks already in blogs. Readers won't use them if they're not interested. So regarding /m42, Kevin, I don't think there's a danger of reinventing Usenet, where threaded presentation came with the package, like it or not. But you raise a good point: "Thus TN is basically a way of 'fixing' the asymmetry of links". Yes, I think it distills down to that.

You follow with "Care needs to be taken here, because this asymmetry is what makes Google work". Excellent point, I think. With every blog thread of note linked to Needly, will Google boost the rankings of trailers-on via the Needly link? Could some bloggers be motivated to follow on to heavyweight bloggers' threads just to boost their Google rankings? That's a (admittedly questionable) positive effect for them, a negative effect for blogthreads in general. I'd take that quite seriously, and suggest that Needly implement robot exclusion rules -- specifically, a rule that links should not be followed; there's benefit to indexing Needly itself.

Edited by author 06-24-2002 06:16 PM
Hey, if Stavrosthewonderchicken gets a pony, I want an owl. (Actually, I owlsat once upon a time, and I have no great desire to do it again.)

And Bb, you're right between wood s lot and Visible Darkness (hence, daily & early)

Y'all are losing me in the details of not following links from bots to boost Google rankings, but I'll take Steve's word for whatever.

As a user, I don't care whether a link is raising someone's Google rank or not; I mostly want to be able to follow a blogthread. So I'll be most interested in having a sense of where each piece fits in the puzzle (which will not be strict chronological order) and in which the keystone posts are (the ones around which the other discussions are whirling, so I can read Shelley on USF before I read a dozen other people saying "Did you see what Shelley said?"). Given those two features, I'll be fascinated in whatever shape the project takes.

Another thing to drop into the breeze: I just emailed a blogger the correction for a typoed link from his blog. The issue of links that are edited later might have interesting implications for Needly. Or may just be headache-inducing.

And another thing I'd love to see be able to happen: On Jacob Shwirtz's blog ( http://www.gazm.org/blogs/2002_06_16_Fuzzy_Blogic_archive.asp ) he said, of another blogger's theory: "I won't try to explain it here but maybe he will on his own blog." I imagined for a moment that, once that explanation was posted, this sentence would then magically be linked to it. But then I reentered reality.

I suspect that such capabilities are outside anything Web based, and only exist, perhaps, on the road to Xanadu. But it would be cool.

I found the Disenchanted links back to you article, which implements much of what I described in /m42

Also, Joseph, they have anotehr article discussing Ghosts of Xanadu and why it failed (too complex, too centralised).

Stavros, per /m44, hopefully once the data begins to accumulate, others will step forward with tools that consume it and provide interesting presentations of same.

Steve, the data standard - RDF specification - is ThreadNeedle; the application is more of a test case and implementation to encourage adoption.

Steve, I'll be proving a first draft of RDF and a walkthrough of how the process works in a couple of days, as soon as I get DSL access.

As for how this works with daypop and google - I can easily restrict the Google bot. I already have restricted it from most of my weblogging site. I'll allow Daypop through as a way of "tracking popular discussions" because they'll show up on Daypop 40. But Daypop, Blogdex, and Google are all well-behaved bots - they'll go away if we tell them to.

AKMA, I hear what you're saying about being able to take in all of the discussion at once, understand the pivotal posts. We'll have to play with this one a bit, but the same data can be displayed many ways.

My hope is that better UI/graphics people will take the data and run with it. Additional hope is that a whole bunch of people start scraping the RDF from weblog pages and we get all sorts interesting variations of ThreadNeedle.

Edited by author 06-25-2002 10:50 PM
Shelley, I'm sure you are thinking about this already, but defining an RDF vocabulary for blogs in general could and should be part of this. Blogs have certain common characteristics, whichever tool generates them, and defining ways to distinguish these from layout is highly useful.
The obvious ones to me are tags bracketing posts and indicating permalinks, tags indicating the main page, and tags delimiting blogroll entries. I'm sure there are more.

Edited by author 06-26-2002 02:19 AM
Kevin, are talking about something like BlogML? It's an idea a number of folks have bashed around quite a bit lately.

Ken, RDF for weblogs would be extremely useful. BlogML isn't RDF, but could be made into an RDF-compliant vocabularly easily enough. With this, weblog postings can be expired, search engines would return more positive hits and so on. Hopefully this will be pursued.

Also, hopefully, Google will finally start to process RDF.

However, going back to my original "Why I'm not using RSS" statement - ThreadNeedle is for a specific purpose - tracking discussion threads. As such the main entity behind ThreadNeedle is a discussion rather than a specific weblog. The scope of focus and the "business" behind ThreadNeedle extends beyond a single weblog. As such, the vocabularly for ThreadNeedle isn't necessarily compatible with the RDF used to describe a weblog.

Personally, I'd like to see RSS merged into a weblog RDF ML, myself.

Sorry. Kevin, not Ken. Still rummy from my trip.

Has anyone else seen this new feature in MT 2.2? Does this change our thinking about ThreadNeedle at all?

Edited by author 06-27-2002 09:18 AM
Shelley, re /m55, you probably know that RSS 1.0 is based on RDF, and so is extensible using namespaces. ThreadsML, for example, is based on RSS 1.0 and specific uses for existing non-standard RSS modules. My proposal is here. I had the same reservation that RSS was just for aggregation early in that discussion, but was swayed towards it, thinking that common parsing tools can be developed (I don't see much on that front yet though).

I think the same might go for BlogML (a discussion I haven't yet followed). RSS 1.0 might be a good basis.

Stavros, I beta tested the trackthread - it really isn't any different from the referrer lists that Mark Pilgrim uses - tracks one level back. It doesn't map the whole conversation.

Personally, I think it works with rather than in place of ThreadNeedle.

Steve, re /m58 - Extending a vocabulary is technically feasible, but you have to define the focus of your business. To me, RSS is a syndication/aggregation vocabularly whose main entity of interest is a specific weblog. Same with BlogML. ThreadNeedle has a dialog as a main focus, and mapping between entries across weblogs - totally different business. I covered this in a weblog posting at http://weblog.burningbird.net/archives/000291.php - if you get a chance, you might want to check it out and see what you think.

More on /m57 - I did chat with Mena on this when she read about ThreadNeedle. Personally, I love trackback and plan on implementing it's use at my weblog. Bottom line, though, is that this is an implementation rather than a specification - which means it's tool dependent and proprietary. ThreadNeedle is first and foremost a specification and then an implementation, non-proprietary and non-tool dependent.

However, what trackback will do is show how this type of cross-blog communication is impacting on users, webloggers, and discussions - could provide good usability requirements for use within ThreadNeedle.

I think Ben and Mena really blasted the technical edge with trackback - excellent move. So is the port to MySQL.

I haven't worked through the formal RDF schema at this time, but I do have RDF test cases - 1 for thread start and one for thread reply.

To get a visual idea of how these work, you can pass these to the RDF validator at http://www.w3.org/RDF/Validator/. In fact, I may use the graphics API from the Validator as the first graphics diagram of a dialog (until more skilled folks take over.

In the RDF, a discussion is also a thread (see start), and the discussion information is redundantly added to each thread - carries through. The reason for this is that a "discussion" transcends any one thread, and should be described separately.

Notice that a thread reply can reply to more than one posting.

There's no reason that ThreadNeedle RDF and RSS RDF can't be combined within one posting - RDF from many vocabularies can be combined (original purpose of namespace). However, for the most part RSS is generated for _every_ posting, but I sure hope that ThreadNeedle RDF is NOT generated for same - ThreadNeedle was meant for more complex discussions, as an invitation to same.

I've just registered threadneedle.org and as soon as the DNS change propagates, will start moving work there. I know not everyone who is participating in this discussion is comfortable with RDF - I'm throwing this out for the RDF folks for validation of syntax. When I get threadneedle.org setup, I'll put out some test pages so you can see how this RDF would be processed.

It's a start.

Feedback PLEASE PLEASE PLEASE!!!!

Edited by author 06-28-2002 12:40 AM
Motivated by the challenge of a tree representation, I cobbled together a tree-based viewer for QuickTopic. Parentage is based on the /mNNN links in a message, and it illustrates the problem of representing multiple parents using a hierarchy.

Here's the tree view for this topic:
http://www.quicktopic.com/cgi-bin/not_ship...t=P96CAYje2yc&n=all

Note that /m37 refers to /m33 and /m35 (and this one refers to all three), so /m37 and its whole subtree appear under each of them. It's a compromise solution. Are there better ways that still use this familiar tree-node display?

[This quick hack was based on Jeff Murphy's Expander.cgi found at http://sware.cit.buffalo.edu/Expander/, and isn't real-world-ready.]

Tree representations are complicated when one is directly responding to more than one response. Originally, I had planned on duplicating the person's response under each item they were responded to and then flagging that the response is "duplicated" in the hierarchy.

One thing with something like this - there won't be a lot of responses as one gets in a mailing list. Usually, there will probably only be 10-20 postings in all. Even with Daypop, top items usually don't have any more than 40 links at most.

Since there are no comments on the RDF used, or the fact that ThreadNeedle is NOT mapped on to RSS, I'm going to assume that I can proceed with my original implementation.

As for displays, the data will be there and hopefully others will try doing something with it.

I think there's some benefit to using an RSS-based RDF, especially in that people might want to use an RSS aggregation tool to track a blog thread, but I'm confident that your RDF can be mapped to RSS.

The other consideration about RSS vs. other RDF is the number of toolkits in various languages that'll parse it, but there seem to be enough RDF tools.

Far more APIs and open environments for RDF than RSS from what I can see.

We have to be very careful that we don't subsume RDF's metalanguage ability into creating one super vocabulary - RSS - that is used to solve all problems. In data circles, this would be like trying to create one database for all businesses, and forcing said business into the model just to make it work.

As for using an aggregation tool to track a blog thread - how would this work when aggregation tools track one level deep? If you think on it, aggregation tools are pretty basic - site updates, here's extract, end of story.

Nothing I can do to stop RSSifying ThreadNeedle, but I won't be sanguine about it.

> As for using an aggregation tool to track a blog thread - how
> would this work when aggregation tools track one level deep?

In /m45 I asked "Is there also a data standard for the aggregated thread that's gathered by the thread-serving site?" That's what I'm thinking of -- a single gathered data chunk representing the entire thread. I should be able to use an aggregation tool to track a blog thread as a single channel, just like I track any single blog. What's your thought on this?

> Nothing I can do to stop RSSifying ThreadNeedle, but I won't be
> sanguine about it.

Could you elaborate? I must be reading "Nothing I can do to stop ..." wrong -- it comes across, well... not very open-standardsish.

Steve, per /m68, you really couldn't track a multi-threaded, multi-level discussion - there is no 'single track' nature to this. If there was, we wouldn't need ThreadNeedle.

In addition, until most aggregation, ThreadNeedle tracks deliberate conversation - someone posts a thoughtful, in-depth posting. Two people read it and create their own postings on same, and other people respond to the original or to others and so on. Think of brain synapses, which spiders out in a complex pattern rather than necessarily following a linear path - and that's ThreadNeedle.

RDF is open specification (not standard - W3C don't do standards), on which RSS is based. The level of complexity necessary to begin to track these complex, cross-blog (other medium, potentially?) conversations goes beyond the linear, subscription, syndication, aggregation nature of RSS - or what RSS was originally intended to be before scope creep came along.

Bluntly, if I put ThreadNeedle out with a preliminary implementation and people grab the vocabularly, break it down and force it into the RSS model, and try to lineralize the very complexity I'm trying to support, then why am I going through the effort.

In other words - RSS doesn't fit, seriously doesn't fit, every business use or usage pattern. That's a bad, dangerous trap to fall into, and one that has me greatly concerned before even starting with this project. I hate the thought that a considerable amount of time is going to be wasted because people see "weblog" and think "RSS" and squish ThreadNeedle into this narrow mold, accordingly.

ThreadNeedle is going to be open source, in Source Forge. And it is based on RDF - but I'd be an idiot if I sit back at that point and say "Well it's out there. Break it."

Wouldn't I?

I'm no longer suggesting RSS as the primary representation. And, yes, a linear presentation of a blogthread would be a weak shadow of the full structure. The best navigation app will show the full network.

I still have this question: will there be single aggregated bunch of data representing a blogthread, available to any app? Or put more open-endedly, what's the model by which other apps will access a blogthread's RDF data? Will they need to spider weblogs themselves, or will there be a published repository? If the latter, will there be notification of updates?

After this I think it's time for me to be quiet awhile. Thanks for all the good follow-ups.

Sorry, Steve, didn't mean to come on so strong.

The entire discussion won't be represented in each block of RDF per thread. To recover the whole dialog an app would have to parse the RDF from each thread. The thread's location in the dialog would be mapped within the block so if "strands" are missing, at least partial representation would occur.

Currently, the RDF shows the individual thread and the node or nodes it's replying to. In addition, I have the topmost node duplicated. I could duplicate the entire thread leading down to the node, but I hate to cram too much data into the posting (though it is only a few bytes).

This is actually what I was hoping to get help with when I posted the RDF examples - to get feedback, suggestions, that sort of thing. However, we kept focusing on implementation. Such is life.

ThreadNeedle will have the aggregated data for an entire discussion, but I'd like to see this distributed - mirrored elsewhere. Regardless, apps will be able to query ThreadNeedle for the data - at least until my server gets overwhelmed. However, I think we'll be okay until the data/app is mirrored.

Again, sorry I came on so strong, Steve. It's I who should be bowing out of this.

I personally would prefer neither of you bow out of anything. I'm only barely following what you're talking about, of course (which is perhaps why others are lurking as well, to see what happens when the tech dust settles), but I find it faskinatin'.

Well, Stavros - Steve hit it on the head - where does one record the entire discussion?

The RDF that I put out can record a thread, or the top node of the discussion, but consumers of the data would have to catch each "thread" going by or part of the conversation is lost.

Now, ThreadNeedle will have all of the information but I did NOT want to build in a dependency on this product/spec - that to me, would be a failure ultimately (though I know it would provide something for us all to start with).

This is a classic example of problems one can have with a true P2P environment - where does one store the infrastructure, the meeting points, the common ground, the golden gateway?

I am open to suggestions on this, and will continue ThreadNeedle, the initial application, in the meantime.

OK, Stavros, I'll nibble again, with Shelley's offlist encouragement too.
Shelly wrote:
> The RDF that I put out can record a thread, or the top node of
> the discussion, but consumers of the data would have to catch
> each "thread" going by or part of the conversation is lost.

I guess I still don't understand what you mean by a thread here: is it the entire conversation to date? If not, what is it? It seems like you might really mean "but consumers of the data would have to catch each
post-within-the-thread going by...". If that's the case, what's the model for catching them? Publish and subscribe? Polling?

OTOH, I should ask how important it is to people that other apps be able to build on this without spidering blogs themselves. For one thing, if Needly is the killer app and it's open source (enabling contribs that continually improve it), how important are presentations by other apps?

Jonathan Schull, founder of Softlock, has blogged about visualizing blogthreads:

http://radio.weblogs.com/0104369/2002/06/29.html#a124

At the risk of revealing yet another un-hip lurker, I'll point at an [Caution! Overlong Alert!] query of mine about the relative advantages of/differences between blogthreading and "simple threaded discussion at a single site," over here in this post in another QT. David Weinberger gives his typical cut-to-the-chase answer (meant in the good way!) in the following posting.

Hm. This comes to remind me that I once volunteered a spec for doing symbolic cross-referencing between QTs (to facilitate cross-Topic dialogs); and what I got was Steve's typical KISS /mNNN implementation within Topics. :-) Elegant.

I mention this because it seems vaguely relevant to the Holy Grail here, which is (imho) a means of visualizing and following blogthreads from a server(s). Seems to me that

1. This depends on the blog-conversationalists' adopting some sort of "I am responding to topic Y in the posts of P and Q and I'm marking it up intentionally this way" notation. Fine -- may be easiest and most powerful such notation get adopted... and that
   
2. This has a chance of working, as long as it doesn't depend on some sort of automatic mechanism for spidering, finding and marking elements of a blogthread. That way lies semantic madness, methinks.

Steve, I see a thread as one post in the conversation. Now one thing we can do is provide the hierarchy from the topmost node in the discussion to the current thread, so that at any one point in time, we have the threaded discussion leading to the one post (no matter how deep).

It's the spidering that we're missing, without having to go back to ThreadNeedle directly, or catch all the posts going by.

As for catching the posts, you can use any technique to grab the RDF from the weblog pages, including looking for any updated weblogs in changes.xml (at weblogs.com) or use a polling technique. However, aren't we all getting a wee bit tired of all these aggregation tools hitting up our files.

What I'd love is a way to distribute the entire dialog as RDF in such a way that ThreadNeedle - the app and data store - wouldn't be a dependency.

Tough.

David, weird, I was just looking at Jonathan's visualization exercise. Would be a lovely front end to the ThreadNeedle data. Best of all, gives me ideas for new data to possibly track.

Dan, I responded to your message at the other QT in that thread.

Cross-Quick Topic. Sounds like a job for ThreadNeedle.

Some notes on issues, progress, some RDF questions at http://weblog.burningbird.net/archives/000319.php

Edited by author 07-02-2002 12:45 PM
Shelley, comments/questions on your page ref'ed in /m81:

The path to G isn't necessarily a unique path. Is it supposed to be the shortest one? What's the purpose of saving this path?

Regarding HTML markup (I think you're citing line breaks as an example, but can't there be other HTML?):
RSS 1.0, if it's worth anything as an example, uses CDATA sections to prevent HTML markup from being processed. (You can see it in QT's RSS). There's been dissussion of this in the RSS-DEV list at Yahoo Groups.

Edited by author 07-02-2002 10:55 AM
Shelley, a little late commentary on the RDF examples you gave in /m61: It doesn't contain the URLs of the blog posts themselves. Are they to be inferred by the spidering tool from the context of the RDF? If so, that might be a problem, since a given blog post is automatically replicated in several places by Movable Type, for example: the main page, an individual page for the post, and an archives page.

Steve, /m82 - Good point about path being unique and probably why I won't include. If the person responds to more than one post, more than one path can exist. Best bet is to just capture the directly reply-tos and the discussion thread.

As for having this information, if a tool was only interested in the direct chain, they could grab this and not worry about trying to capture the other bits of the discussion. Mainly a convenience.

Per /m83 - this is fake data, but the URLs of the posts are in the examples. When I test the tool, I'll grab one of the more detailed discussions out and about as a test case so we're working with real data.

When the person registers their post as part of the discussion, they'll provide the permalink to mark the post - whatever this is defined to be for their weblog. The only thing that matters in this circumstance is permalink.

Of course, nothing to stop from using any other URL - this is a manual operation.

Also per /m82 - the markup that the weblogging tools generate is line and paragraph breaks (<br> and <p>) and so on - and these are generated around the RDF tags - CDATA doesn't do a bit of good. Remember that CDATA is included within RDF tags, not surrounding RDF tags.

I can surround the contents within CDATA when the RDF is generated, but can't stop the weblogging tools from generating the HTML.

In the QT example, if this was copied to a weblog posting, the weblog tool would put a <br/> after every line in the RSS. This will cause any and every RDF processing API to break.

The whole concept of embedding RDF directly into an XML or XHTML or HTML document (not the header, the body itself) is very new, and very experimental. Which is unfortunate because this makes sense. As it is, I had to surround the RDF with a script tag to get the HTML processors to ignore.

Ah, I see the problem about <br>s now. Thanks for the clarification.

Yech, that's a hard problem. It makes me want to re-propose (/m20) simply using a text indicator adjacent to a link. Your reservation in /m26 was "The main reason I wanted to embed RDF directly into the post is so that other agents can consume it. I was hoping to eventually not have the only ThreadNeedle application."

If threadneedle.org could become a central repository for the RDF for full conversations, that would give other apps the necessary access, and would also make for fewer spiders running around my blog (and requiring expertise to do so). Is bandwidth usage a concern?

Oh, how I wish I had heard about this thread earlier. I don't feel quite right jumping in 90 posts into a conversation, but I think I have some ideas that people may find useful.

When I think about blogthreads and blogthreading applications, I think in terms of three "layers".

Layer I is the weblogs themselves. Archives, main pages and so forth. This is the stuff served in HTML pages and stored in files or databases.

Layer II is the graph of posts. Every weblog post that has a URI (a permalink) is represented by a node, and they're connected in various ways. This is the sort of thing RDF is really good at, so it's helpful to have an RDF vocabulary to describe the posts and connections for interoperability. At a minimum, the ability to say "Post X refers to Post Y" is all you need to describe a threaded discussion.

Layer III is the applications of the graph. This is how users will experience the connections present in Layer II. Given even the minimal "refers to" expressiveness, one can do thread overviews (for a given post, show a set of posts which refer to the chosen post, and a set of posts which refer to them, and so forth to some depth) and "forward" links (a set of links associated with a post showing other posts which refer to it; this can be presented by the weblog itself, by a third-party website, or by the browser).

Generating the Layer III applications from the Layer II graph can be complicated, but it's nothing new. Message boards and usenet readers already solve similar problems. Anyone who has or can generate the graph can make an application to display it to users along with whatever innovations they wish (filtering, Google-style ranking, etc.).

Getting the Layer II graph from the Layer I weblogs is trickier, and there's bound to be multiple methods. The neat thing here is that there's no requirement that all the data in the graph be gathered the same way.

In /m9, Kevin Marks writes:

We need a way to detect the span of a single blog post, and identify an incoming permalink to it. Then we can make a spider that crawls blogs and shows the threads implicitly.

I describe one way of doing this in http://www.eyrie.org/~zednenem/2002/web-threads/ . Essentially, you put a special div element around the post to mark its span, and any link inside the div that goes to another post is a reference. There's even a way to identify a permalink using the "rel" attribute.

I recently retrofitted my weblog's archives to match that coding convention, and I was able to generate a list of all the "threads" within my blog with a fairly simple Perl script. It's at http://www.eyrie.org/~zednenem/2002/web-threads/thr.html . The whole thing is generated from reading the HTML code (except for the dates, which I derive from the URL); there's no CGI or content-management stuff involved.

Steve, per /m86, never ask a P2P person what's the harm of having a single centralized app ;-) Seriously, I'll implement (well, am implementing, must blog less, develop more) ThreadNeedle as a centralized service to start, but won't be content until it's decentralized.

Edited by author 07-03-2002 08:48 AM
(tangent) Just to backtrack to the UI stuff again, have a look at this. Sna-zzy. It's Open Source too! (/tangent)

Carry on...

Very cool, Stavros! Thanks.

Again, I think I'll lay off awhile here and let you, Shelley, get to the fuller examples, user walkthrough, and of course implementation. I advocate iterative development -- there's nothing like a working example to flesh out the issues.

Early on, there was discussion of what kind of meta-information to track, and discussion of referrer information (which is indeed interseting). But I'm interested in something else, too.

How did a blogger become aware of the item he is following up on? If he is subscribed to the item's author's blog, we can guess. But if he is not, this is something only the blogger will know; s/he should be encouraged to record the fact in a way that can be extracted by your tool.

Regards,

Edited by author 07-04-2002 01:40 AM
Hey, I can't believe that I didn't know about this discussion ealier. I am reading through all the posts, and hopefuly will contribute soon!

I have been approaching the subject from the point of view of having visual forums where each post is a node in a graph. The point here is that users could reply to multiple messages (and those connections would be made explicit by showing edges between the post and the messages that were responded to). Also, one could make a passive change by voting to strenghthen/weaken the tension in any edge (weakened edges would eventually disappear).
  
For more, please take a look at my followup comment to http://nooface.com/articles/02/02/04/1830239.shtml

Hi Jon, not sure of your question - could you provide a bit more detail?

Alex, interesting concept of voting to strengthen or weaken the thread to a post.

I have a question for those interested in UI aspects of spec - if you could define the service that ThreadNeedle would provide for querying data from external tools, what kind of service would you see?

(i.e. how would you invoke, and what would you get and in what format?)

Edited by author 07-05-2002 04:10 PM
Re /m94:

The service I'd like to see is the ability to ask "What posts reference this post?" This could just be a simple web service that returns a list or links or a bit of RDF in response to a query. For example, http://thread.org?refersTo=http://example.org/blog/20020703%23p4 would be a list of posts which refer to http://example.org/blog/20020703#p4 . (The format of the response could be plain text, with a URI on each line, or RDF or something else.)

Two user-level services that third parties could offer are weblog mediation and thread overviews.

Mediation is the sort of thing Crit does. You would ask it for http://thread.org?page=http://example.org/blog/20020703 and it would go grab a copy of http://example.org/blog/20020703 and after each post it would insert a links to posts which come after it in the thread. (Edit: After creating the modified copy of the page, it would send that to the user.)

For thread overview, you would ask for a page such as http://thread.org?start=http://example.org/blog/20020703%23p4 and it would show the thread following http://example.org/blog/20020703#p4 . (What "the thread" means depends on the algorithm thread.org is using. It could be forked or linear or some sort of graph.)

Edited by author 07-07-2002 10:45 AM
Hi Folks,

I just recently found about about Shelley's ThreadNeedle project. It sounds great!

At essentially the same time as this QuickTopic's been happening, I was independently coming up with ideas about Weblogs and cross site conversations, many of them very similar to the ones mentioned here (darn, I thought I was totally original!). I wrote up my ideas at http://www.dancingrabbit.org/skyhouse/webconversation .

I welcome everyone taking a look at what's there. I'm not attached to the name. I am into the concepts of widely mirrored conversations with signed content.

I'm not sure whether ThreadNeedle as it's currently envisioned could encompass the concepts I'm interested in, but I'd love to collaborate with other people to create a good standard that would be widely applicable.

Spoke too soon about resources on server. I am going to have problems unless I upgrade. Ouch.

Welcome new discussees.

David, thanks for feedback on queries per /m95. Service should be able to return RDF based on a specific URL, given that the URL is considered the start of a discussion thread. To start at least (resource issue speaketh again)

Server upgraded and work progresses.

Work is continuing at threadneedle.org. Once the initial prototype is finished - hopefully in about a week - I'll post a note to this discussion group.

The prototype will give us something we can all work with, improve.

Hack and trash, in a nice way ;-)

Needless to say, I'm running behind. Another week or two.

Hi Shelly, good to hear that your coding as well as your writing continues.
I also got carried away in my initial excitement at seeing this
discussion. However, I remain content in the knowledge that making connections between blog threads explicit should become a reality one way or another. Isn't that sort of what instant outlining was about? Best link I could find: http://radio.weblogs.com/0100887/2002/03/30.html#a157

Yes. Still behind. Sorry.

Teoti.com - And you thought you were going outside today -http://www.teoti.com

Discussion about ThreadNeedle, data, and other issues is also ongoing over at BlogMD at http://www.truthlaidbear.com/blogmd/forum/index.php.

Edited by author 09-06-2002 01:43 AM
Status of effort:

My initial implementation plan was to have users access a web site to register a discussion thread, grab generated RDF from this site, and post it into their posting. This would include RDF/XML decribing the discussion thread, as well as links to ThreadNeedle to generate RDF for use by others wanting to reply to the specific posting.

Well, sounds good but has problems.

First problem is, as has been discussed, RDF embedding into HTML bodies is a problem. The processor will display data contained in elements and the only way to avoid this is to surround the RDF in a script tag or to use such simple RDF that all of the relevant data can be described as attributes of an element. Regardless -- using the RDF will prevent the web page from validating.

Still, this will work, except for a second problem -- weblogging tools want to add white space HTML to all content, and if you past RDF into the posting space, HTML breaks will be added. This will break RDF parsers (the XML is no longer valid RDF at that point). The only way around this is to no have line breaks in the generated code (not workable) or to turn on automatic line breaks (not feasible for most webloggers).

With the release of MovableType's TrackBack, they did highlight another option I hadn't considered (too close to problem, too dense, or both), which is to embed the RDF into the template rather than the posting, and use the weblogging tool's template tags to provide the relevant data (URL, title, author, date). This is great, except for one thing: ThreadNeedle isn't tracking postings, it's tracking dialog threads.

A dialog thread is one person writing a posting about another posting, providing link to same. Sounds simple, but there is nothing that tracks these threads outside Daypop or Blogdex, and they only track one level's worth.

I can embed RDF into the templates and this works great, but there is no way to embed the fact that a posting is related to another posting.

So, I have and am looking at yet more options. Among these is just plain scanning weblogs for links, keeping a record of the URLs and then gradually build the dialog when new postings come in that contain these links. This is how Daypop works, except we would show linkages beyond one level. This approach is very workable, but it will be data and space intensive -- too intensive for my machine and I can't afford another upgrade.

To be honest, I also don't like the idea because it's just plain inelegant. ThreadNeedle is for tracking dialog threads, not postings. It doesn't really care about individual postings. The data store would be full of postings that aren't part of a dialog, taking up space. Even with a garbage scheme, the solution is inelegant.

There's another simple solution, technically, and that is create a form, have people input their posting URL and the URL they're replying to, forget all the folderol and just show the discussion. This is trivial technically, except that we're centralizing the solution, which I wanted to avoid, and people have to add discussion threads. I'm finding that webloggers really don't want to have to manually screw with things. Their interests fall considerably if the process isn't automated.

Another problem with this goes back to AKMA's concern about garbage being introduced. As you can see from this Quicktopic, with a certain teoti, anybody can attach anything. This happened to me with trackback when someone, accidentally I believe, did a trackback to one of my postings from theirs that had nothing to do with what I wrote.

The earlier discussed approach of scanning for links would prevent some of this bogus dialog stranding because you would have to have a physical link in the posting for the dialog.

I thought about piggy backing the technology on to Ben & Mena's Trackback (Movable Type at http://www.movabletype.org) especially with their new stand alone server. But again, there is some complexity associated with this for the webloggers, more than I think many are going to be comfortable with.

Another area of concern with this technology is it does require CGI on the webloggers site, which won't work for Blogspot (and other webloggers without CGI access). We could host a server for those folks, and then working on utilizing the TB pinging. An option.

(I've been chatting with the BlogMD people at http://www.truthlaidbear.com/blogmd/forum/viewforum.php?f=1. They're facing many of these same problems.)

Sorry, I wish I had something I could go "fini!" and throw out to the world, but I've about exhausted all my ideas and technical bag of tricks. So I'm throwing this out to the discussion group to see if anyone can spot something that I'm too tired or too dense to notice. Otherwise, outside of seeing what I can leverage on Ben & Mena's effort (and kudos to them in their generosity in making this technology available freely), I'm fresh out of ideas and ThreadNeedle ends with the RDF schema and no implementation.

A workaround for the 'pasting RDF' problem would be for your form to give a javascript URL to paste into the blog that references a javascript on your site that generates the RDF, just as the googleboxes at http://www.edazzle.net/googleboxjs/googleboxjs.asp do.

On the spider front, you could define some bracketing xml tags to go around a post that contains a link to be considered a blogthread, and have blogs opt in so you don't need to spider the world if you don't want to.

Thanks for the suggestions. Unfortunately, the RDF was normally to be pasted into the posting that would contain the form element you mention.

I'll look more closely at your XML tag suggestion. Thanks.

I just noted this via Dave Winer's blog:

http://www.myelin.co.nz/cgi-bin/wcswiki.pl...slyEasyGroupForming

I added my own reference to this discussion there, with my own bias towards my own suggestions here. Feel free of course to add your own, and to modify the BlogThreads definition I put there (it's a wiki).

Anything new here?

Hi Steve

I've basically incorporated Threadneedle into my PostCon (Post Content) application. You can see an example RDF file from same at http://weblog.burningbird.net/chap0606.rdf. First release of the app will be built partially in Perl and partially in PHP. Second version of the app will retain the Perl bits, but replace the PHP with Java.

Note references and is referenced by statements. This is a Threadneedle 'level'.

A file such as this is generated for each of my web resources, either manually through HTML forms, which generate the RDF; or automatically through something such as an MT plugin I'm working on. One file is generated for each individual posting archive page.

The reference statements are pulled from the content, or again added manually with forms. In Movable Type, these are marked by specialized tags that are processed when the posting is published. The referenced by statemetns are pulled from HTTP referers ( or through trackbacks as a possibility).

When the RDF page is generated (but post publishing to not hold), a small bot follows the references URLs and accesses the PostCon RDF page for that page, if one exists. If it does exist, the references are accessed for that page and the bot records the info and continues the traversal, recording all 'references' it finds. This is recorded into ThreadNeedle RDF file generated for the posting/page (if the level extends beyond one level). Additionally, the bot follows the referenced by links as they are pulled in and, again, looks for associated RDF pages -- it would grab the referenced by for that page and follow it and so on. This information is added to the ThreadNeedle RDF file.

My PostCon application has a RDF-to-XHTML pretty printer that takes a PostCon RDF page and prints it out. This is linked to automatically from each web page (click info link, this page opens). The generated page contains a link to a ThreadNeedle page if one has been created (this becomes a new property in the PostCon RDF page). The application also has an associated, but separate RDF-to-XHTML pretty printer for ThreadNeedle RDF content. The page will then contain the ancestors and descendants for any PostCon managed page.

The only thing left at this point is a bot that looks for these ThreadNeedle 'thread' RDF pages, and builds the entire conversation. Not complicated -- one can use weblogs.com or bl.ogs, or just plain wondering bots to look for these. However, a database will most likely be needed to pull in entire conversations.

This all presupposes that each referer and referenced by has one of the PostCon RDF pages for every posting (or web page). With tools to generate these, though, that isn't necessarily a problem. Would just take time, and acceptance.

Anyway, this is the approach I'm working on and one I hope to finally finish by end of this month. I'm using bits and pieces of the code in the RDF book, which I have to finish next week.

One final note: once the PostCon RDF pages are generated, any app can use them for any purpose. Same with the associated ThreadNeedle thread RDF page. My hope is that we'll see as many interesting thread readers and aggregators as we do RSS thread readers and aggregators. The premise is identical, except that ThreadNeedle is multi-level, which RSS is not.

Edited by author 12-12-2002 10:53 AM
Thanks for the update, Shelley. I'd like to understand this better from the user's perspective. This sounds like a combination of the approach of /m15 (the user adds a tag before a link to a blogthread post and a crawler picks it up and builds a blogthread database) and the RDF-based approach.

In this case, it's still fairly simple for the user, who adds specialized tags to a posting, which are processed automatically by your MT plugin (or somehow processed using "HTML forms" -- can you explain that more?), to create RDF on the blog author's site, which is picked up by a crawler.

So it sounds like it has the relative ease of the simple approach, combined with the standardization of RDF, at the expense of one-time installation of an MT plugin or periodic manual processing via "HTML forms".

One question on this: "Additionally, the bot follows the referenced by links as they are pulled in and, again, looks for associated RDF pages -- it would grab the referenced by for that page and follow it and so on." Why does this need to be done each time? Is it just for robustness? It seems that the referenced-by nodes would already have been spidered independently. [Correction: the *references* nodes would have been spidered independently. But that raises another question: if referenced-by links are built from HTTP referrer links, what if no readers have followed the link yet? It seems your central thread-builder could have the smarts to build the relationships.]

My biggest concern is that it sounds like usage has to be unanimous in order for this to work. What happens if a link in a blogthread isn't using PostCon or hasn't run a necessary manual process to create the PostCon RDF? If using PostCon is mandatory, it had better be dead-simple to install and use. Even then, some people won't use it.

Hi Steve

Yup, you got. And a bit more clarification how this whole thing pulls together (PostCon):

PostCon is really a set of small, lightweight apps that can be used independently. No large framework, no download bandwidth busting and difficult to install and configure app. Use one piece, use them all.

If anything, PostCon is the RDF file -- the apps are nothing more than processing sugar.

One piece is an MT extension that generates a PostCon RDF file for the post when the post is published. And a link to the page is put into the head section for the posting individual page archive. (this is a one time template change -- easy).

This MT extension (integrating into app is still a tiny challenge at this point) will pull the information for the PostCon RDF page from the posting data, including any hypertext links it 'references'. Once the PostCon RDF file is built the MT extension app is finished. At this time, no ThreadNeedle file, and only one level of conversation is recorded.

Now, another app will access this RDF file, access the referenced property and traverse the referenced URL to look for more referers (drill down). If it can't find a PostCon RDF file, it stops. If it does, it continues until it finally hits an end. If an additional level to the conversation thread is found, the bitty bot builds a ThreadNeedle RDF file, automatically, and plops in the info. One threadneedle file for one PostCon file.

Another app accesses a combined format Apache log file for referers and plops them into the referenced by property for the appropriate PostCon RDF file. That's all it does -- listen for referers and update.

Another app (same as the bittybot mentioned above, but given a diff RDF property target) accesses the referenced by url and drills up. Same limitations with drills down -- no PostCon RDF files, it stops. It adds the new info to the ThreadNeedle RDF file.

So far, each is a small independent app which can be easily replaced -- the key is the PostCon RDF file, and the concept of following a specific thread. Another key is that the bot won't try to spider. It will look for PostCon RDF, traverse vertically, and that's it.

Now, another app could be written to follow the ThreadNeedle RDF files, and do the spidering. Much of its job is done by this vertical processing. However, all I'm concerned about at this point is capturing one single thread.

I'm hoping that by keeping the processing simple and stupid, which makes it lightweight and incredibly easily tweaked, propagation of PostCon RDF will occur quickly. And there's nothing more that techies like to do then tweak, and I'm giving them perfect tweaking material.

(BTW, this is all going in SourceForge as open source.)

Continuing into PostCon outside of ThreadNeedle -- another application is my XHTML web page app that generates a PostCon RDF file from manual entries. I'm using this on my existing web sites pages (not the weblog). That's all it does -- maintain the PostCon RDF file. This includes updates if the web resource moves, or is removed or replaced.

The core functionality of this will be exposed as an API (PHP at first, then Java, and then Perl), and will be wrapped eventually in SOAP/XML-RPC. So one could incorporate this functionality into exist apps, such as weblogging tools, content management systems, and so on.

And I have another tiny app that takes this information and adds to a 404 custom error handler (this is already in use at my site) datastore (hashed file), adding a reference to the new location for moved content, and to the RDF file if a page is deleted or replaced. When a 404 occurs, my errorhandler redirects links for moved pages to the new location automatically. It also calls my rdf-to-xhtml pretty printer for a replaced page, resulting in a very detailed info page that provides information about why the page was pulled, and recommendations for replacements, and so on. This is the same pretty printer (again, a separate app page -- one single php page) that prints out the page information if a person clicks the 'info' link I referenced in my earlier message.

(I feel like I'm selling a Sledge-o-matic, but one other app I'm hoping to have done by end of Jan is using Mozilla and composer and MT's SOAP access to create a desktop WYSIWYG app for MT. And because it's Mozilla, it's ported to all the Mozilla environments. Info not to go beyond this list at this time, please.)

All separate small, open source, easily tweaked, prime time language applications -- the most extensive app is the PostCon manual RDF entry application and that's dirt easy to read. A developer or user could use any one piece alone, or in combination.

There is one config file that pulls this all together if someone wants to use all together if someone wants to use the whole gang of apps.

Nothing fancy. Simple. No brain science stuff. And based on the premise that PostCon RDF files will become ubiquitous, at least among the virtual neighborhoods that want to use this.

(Of course, once this all goes open source, better developers than I will increase functionality and add good stuff. I hope.)

Edited by author 12-12-2002 11:46 AM
I'm a little lost among these small apps. Does "Use one piece, use them all" mean "if I use one, I'm using them all" or "you can use one, several, or all of them"?

What do I, the (usually non-technical, remember) blog author, have to set up and use to get blogthreads working for me?

Sorry to repeat: What happens if a link in a blogthread isn't using PostCon or hasn't run a necessary manual process to create the PostCon RDF?

Good questions, Steve.

How's about I finish the app (because I really do have a book deadline next week) and then I promise to post to this thread, and to you directly with references to the actual app as soon as it's done. It's so much better to work from real than vapor, and I've promised this functionality for some time now.

Short answers for now:

You can use one app, two apps, all apps (or should I say components?) -- up to you. There is no dependency except the existence of the PostCon RDF file (and the ThreadNeedle RDF file if a person wants to incorporate TN into their pages). Very similar to PHP XML at Source Forge (which I'm using).
If you're an MT user and you want to generate PostCon RDF files, you'll need to drop a couple of files into your MT installation, and you'll need to add a couple of lines to your individual posting archive file. The issue I'm working with now is how to call the functionality to generate the page -- MT plugins tend to be somewhat read only.

If you want to add ThreadNeedle support, you'll need to drop in a couple of more files, add a couple of more lines to your template, and manually edit a config page. You'll also need to create a CSS style sheet and template for the pretty printers (default ones will be given). This can be done through MT for MT users. The HTTP referer app will need a little additional config. The bot is also included as part of this installation. However, you can access the referer app and bot sourcedirectly if you want (only likely a techie will do this.)

All total: about an hour for a non-techie, much less for a techie. One time setup.

If you want to use PostCon's web-based application for manual PostCon RDF generation, you'll need to download a zipped file and unzip it, and you'll need to password protect your app directory (I'm not building security into first release). Let's say about the same amount of time or less to setup a MT blog, and about that skill level. If you want to incorporate an info link into your web pages to point to the RDF-to-XHTML pretty printer, you'll need to do this manually, or add it as part of a template for a MT type app. The pretty printer has a default look and you'll need to customize if you want something else.

If a referenced URL points to a site that doesn't use PostCon RDF, the bot stops. This is no different than an aggregator not being able to aggregate a site that doesn't support RSS. The bot doesn't try and parse the HTML or anything like that looking for links. I'm not building mighty bot, I'll leave that for someone else.

There's room for HTML-to-PostConRDF scrapers and smarter bots that will search for links et al and XSLT transformers et al.

Shelley

>
>
< replied-to message removed by QT >

Thanks for the further explanation, Shelley.

I have to say I'm concerned about the fact that this requires a non-techie to do an hour's work, and that if not all thread participants do it, the thread breaks. I hope I'm proven wrong, but it would be contrary to my experience with social apps: if a new technology must be used unanimously to be effective, it's very hard to get the necessary momentum.

For this reason, I'd like to at least allow for a backup approach: a "mighty bot" searches the blog content itself for the special markup (the one you'll use to generate the PostCon RDF) and builds the thread. So the smarter the bot can be, the less the average user needs to do technology-tweaking (not everyone gets the same thrill from it that we do :-).

One more question having to do with the special markup: how will readers of blogs know there's a blogthread?

Feel free to defer answers until your book is ready (and good luck with that).

Steve, this is a modular app -- anyone can write a mighty bot if they want, pull stupid bot, put new bot in.

If people want PostCon and/or ThreadNeedle functionality, they're going to need to read the docs in how to use it, and install it and any necessary libraries. That could take up to an hour. If people don't want the functionality enough, then they don't need to use it. I'm not being defensive here -- just very matter of fact. I'm learning that sometimes you can't please the people no matter what you do, so don't waste calories you don't need to.

Making the bot smarter won't make the install easier. You still have to install the apps, read the docs, make some tweaks.

Now with the SOAP interface and open API, no reason an app can't incorporate the use of basic PostCon/ThreadNeedle functionality; apps such as MT or Radio or whatever; then the users won't have to do anything other than installing the outer app. But I can't tell Ben and Mena or Ev or Dave that they have to incorporate this.

(It wouldn't take much for anyone familiar with the blogger api and Radio's stuff to be able to wrap this. Still, someone will have to download it, install it, and read the apps for how to make it work, wrapped or not.)
As to the question about how will people see this stuff? There's two specialized RDF-to-XHTML pretty printers (one for PostCon and one for ThreadNeedle) that someone calls passing the appropriate rdf file as part of the URL. These pretty printer pages (PHP to start, then JSP, eventually CGI) display the Post Content information or the ThreadNeedle thread. How this page is called is up to person -- they can add a link within a template (the rdf file will have consistent naming based on web page with addition of pstcn or thrdndl qualifiers -- 000345.htm, 000345pstcn.rdf,
000345thrdndl.rdf using MT scheme.)

No reason why you all can't modify RSS files to add this in, except that it's outside the scope of RSS. Still have to have bots to traverse the URLS, but it might be easier to get people to modify their RSS templates. If no one uses the ThreadNeedle functionality of PostCon, I'll probably just pull it someday. But it's the PostCon app and file that's of interest to me -- ThreadNeedle is just another consumer of the data.

Still, I'm pretty sure I'll get users of the app. It's not going to be that difficult to install -- no more difficult than putting in a MT plugin.
As for your comment about a bot looking for the markup in the file, some confusion here, which was my doing. I'm not modifying the HTML of the source of the weblog posting or the static web pages. I'm building a separate RDF file. I'll be using MT tags to build this file, possibly, (might be using the MT API directly), but not embedding anything other than links (link to the associated RDF file, and a link to the pretty printer to see the PostCon data if they person wishes). There will be no RDF or markup contained in the XHTML or HTML page, per RDF group recommendations, and per XHTML validation specifications. If you scrape the page, you'll have to use a traditional HTML scrapper, find the post or relevant material and pull the links.
>
>
< replied-to message removed by QT >

> If people don't want the functionality enough, then they
> don't need to use it.

The thing is, I think we (blog authors) are assuming that it's really the *readers* of the blogs that want the blogthread function. And these readers are foiled if any of the bloggers in a chain doesn't use it.

> As to the question about how will people see this stuff? There's
> two specialized RDF-to-XHTML pretty printers (one for PostCon
> and one for ThreadNeedle) that someone calls passing the
> appropriate rdf file as part of the URL. These pretty printer
> pages (PHP to start, then JSP, eventually CGI) display the Post
> Content information or the ThreadNeedle thread. How this page is
> called is up to person -- they can add a link within a template

It sounds like blog readers will usually need to click a link to separate pretty-printed XHTML page to see if there are any blogthread links. Yikes.
I'm monopolizing as usual here. Any comments from the other seven subscribers?

Steve, I think this may be a feature, not a bug. If a post in a thread is pathological (trolls, flames), it stops propogating. Threads die when participants stop participating.

Individual-driven moderation is a core difference between blogconversation and usenet.

This may have an opposite result from usenet anarchy. Usenet's openness makes conversations vulnerable to its most aggressive and crazy members. Blognet's decentralized control may make conversations more controlled, but more closed, because it's easy to ostracize pathological posts, or posts expressing strong differences of opinion.

More like village, for better and worse.

I haven't yet read all the way through the posts on Shelley's proposed architecture to understand exactly how this would work.

Shelley, does the blogthread engine capture only trackback entries or can it also be configured to pick up comments?

Could a comments engine be modified to support thread discovery also?

- Adina

Adina, I agree that something that approximates real-world social networks, in that a disrupter can be ignored by the group who adhere to their conventions, is useful.

But that's not the problem I see here. My concern is that someone who'd ordinarily be welcomed into the conversation can break the thread by not adopting the technology. Even people responding to her who use the technology can't repair the thread -- every link is important. So in a situation like this, I think it's essential that the technology be extremely easy to adopt for the non-technical person.

Edited by author 01-12-2003 04:26 PM
Reasonable concern; a tool that takes a savvy person an hour to install probably won't get critical mass adoption yet.

The LazyWeb seems to be working pretty well, though.

* Nice plug-ins get integrated into MovableType, Radio, and other tools.
* Hosted services help those who want less complicated software they don't have to install.
* Bookmarklets provide one-click access to blog features.

If ThreadNeedle works, even as a demo for the cogniscenti, the LazyWeb will move the project forward.

"With enough eyes, all features are shallow" -- Clay Shirky

Steve suggested that I crosspost some thoughts about the differences between the the conversational model (ThreadNeedle) and the topic-based model being discussed in RidiculouslyEasyGroupForming Networks.

They are both good, they aren't the same, and they have different strengths and weaknesses.

Threadneedle is better for aggregating a human conversation, whose topic meanders under a named thread.

Topics are great for aggregate blogs that assemble posts about city events, or hiking trails or other specific subject.

A topic-focused blog won't get you a human conversation (that would be ai-complete). A human conversation won't get you a subject-organized index (not without editing after the fact).

The original post and some more thoughts on the topic are here.

Steve, Adina

Currently ThreadNeedle is just the immediate levels of a trackback. Pulling in their associated comments might be too much, but it is a good idea. For this to work we need to be able to access the comments for a particular thread without having to scrape the HTML. We probably could use MT's code, but a better approach would be to have comment's perma=linked, and then return comment links with the RSS associated with each person's post.

It's a start, and what we have now requires little or no work on the part of the person having the conversation, which is what I was hoping for. Still, there is a requirement of TB support.

Adina, agree with your assessment on topic and threadneedle. One is subject, which can have many conversations; the latter is a conversation, which could range over many subjects (usually only one, though).

test bugaga

 
Hallo.! Gutes Neues Jahr 2008.!

Article Opinion <a

She made the sign teen peeing more leisurely pace.
Then her fingers circled strap-on anal any other gifts?
She felt such peace, fetish porn she worried.
Our men too, sighed best boobs that telling fact.
She will be the lolita bondage story toward him.
Nathan started to pace vietnamese porn surprised by the question.
We can then carry deauxma anal that reaction, however.

Hello. I am DruMotana
 
I am like this forum. Excuse me me for my emotions!

They continued to flock celebrity see thru door was kicked open.
He held a silver hot asian girls we must ride.
She tried to capture anime blowjob his earlobe fascinated her.
A woman pulled one, blowjob facials to sleep alone.
On the top sheet sexy teen lesbians with their gold.
The master is in lesbian granny made him hard, aching.
The council members grunted bbw facesitting into tears.

And yes, you have drawings of lotus blossoms face, she told herself.
Kate refused to look reviews on acura tl type-s resisted annihilation?
This was his first ohlins shock bmw r1100s through earth and stone.
I was neither a toyota rock crawlers than three minutes remained.
Lord, she was wonderfully vintage toyota weatherstrip to over seventy.
The man could be 2001 chrysler pt cruiser evap leak detection pump broken windows.
She was laughing when porsche carrera sun glasses the while.

Приветствую всех!
У меня такой вопрос,кто что интересное подска ет буду признателен.
Мы с друзьями собираемся поехать в круиз п просторам России и ближнего зарубежья месяца на два на своих ма инах,но не как не можем согласовать маршрут,если у кого уже был пыт такого путешествия,может,что посоветуете.Девчонок с собой не берем,думаем,что во все городах России с этим не будет проблем,е ли у кого будут рекомендации и в вопросе отдыха с девушками тоже буду признателен.

               С уважением Сеньчик

He trembled his own young models photos on something.
Apollonius was fond of stacy model too inviting to refuse.
Getting a straight answer young models pic them proper farewell kisses.
Then he suddenly jerked sandra model rapidshare de files rapidshare com files megaupload com sin, so be it.
Now, if we could role models and then she stopped.
The flames paled in ff models mean-hearted to suit her.
We probably would have little melissa nude the age of reason?

testovaja probejka