Edited by author 04-27-2006 09:07 AM
That makes a lot of sense as wikipedia seems to be updated just about every few minutes.

I finally got my code requesting and parsing pages from ifsdb so now I have to learn to parse and clean up wiki entries (which are untidy code layouts IMHO) so that I can first:

a. Dump the table that I want in the format that I want
b. Use my code to update the notes field with my now working code.

I also wrote a MySql query passer (yes, passer, not PARSER, as it does no checking) so that I can dump data from tables fast from MySQL server and then straight to disk in the format that I want. Easier than using the interactive terminal mode!! And if it weren't for the over 1 meg MySql dll, my code would be around 25k compiled. I love my compiler!!!


I am having fun and learning a lot. Thank you Al and Rafe for your kind help and your patience.

This query will give the author fields and the note in a single query:

SELECT t1.*,t2.note_note FROM authors AS t1, notes AS t2
WHERE t2.note_id=t1.note_id AND t1.author_canonical='Robert A. Heinlein';

It should be noted ( :) ) that the notes field is somewhat depricated - it's not displayed in the ISFDB anymore as we rely more on Wikipedia for that data. So the notes that you find probably haven't been updated in a couple of years.

Sorry to be a nuisance again but could any of you show me how to do a nested query to use a third form normal search? I can do single form but don't know how to nest queries for third form.

Select * from authors where author='jon grisham'

I.e. How do I get the note data from the note table for 'John grisham' into that query as part of the returned data since the note field in aithors is a link to a third normal form?

Yes, I will be able to learn from the wise masters:):)

Edited by author 04-18-2006 08:38 AM
The database will be static, a reference only. I won't actually be undoing the linksin the original, just creating a table for myself from the database that will include data in the fields that now only includes links. Will be doing lots of studying!!

Before you undo the links, it's probably best to understand why they are there - there are very good reasons for using links rather than the actual data. It may of course be best for your purposes to do without them, but it's a good idea to know why you're doing it.

If your database is static it may well be OK, but if you are updating it then it is *much* easier to keep it consistent if you use the links.

I don't know how much you already know - if you're not familiar with the theory searching on "third normal form" will find you a lot of explanations of this.

THank you both for your help and advice. As I lack the experience and expertise to take this further, i'll stick to the way I am doing it now. At least it works:):)

Eventually, I want to convert the offline databse to sqlite again and merge various tables so they are full of data instead of links. First, I have a lot of learning to do!!!

Still, my little library book manager is beginning to look less useless than it was (ROFL)

Actually, if you're parsing the generated HTML pages, you'd be better off using the XML-based REST API... which isn't fully constructed or public yet. But if you have a specific API in mind, let me know and I'll whip one up that you can test out. (I currently only have two apps - one that returns publications using ISBNs, and one that submits data changes).

As Rafe pointed at, title.cgi uses the title_id as its argument, as that is 1} unique, and 2] immutable (title names *do* change over time as they are edited). If you want to obtain a list of records by title name, it would roughly be:

def SQLloadTitle(titlename):
        query = "select * from titles where title_title='%s';" % db.escape(titlename)
        db.query(query)
        result = db.store_result()
        title = result.fetch_row()
        results = []
        while title:
                results.append(title[0][0])
                title = result.fetch_row()
        return results

Which would return a list of all title_id's by that name. This would not be adequate, as authors sometimes name their collections after a particular short story (for example "Bears Discover Fire), it would also include reviews of the title in question (for example Greg Egan's "Distress").

As far as I cal tell it isn't possible to do it that way.

You would need to have an existing program on the server which would take the title and show that page, and there doesn't seem to be one.

I can't be sure of this though as I'm not clear how recent the code copy I downloaded is, and I never actually got it running. Plus I don't yet speak Python - I'm writing a similar setup in PHP and looked at the ISFDB for general ideas.

However I looked through the files I have, and none of them seem to do what you want.
title.py calls:
SQLloadTitle(title_id)
Whick does select * from titles where title_id = title_id
So it won't work with the text string. ea.py does use the text for the author name.

Unless there's an extra program somewhere or I've missed it on the quick look through I did, that functionality just isn't there. If you can't go via the author page, the only way I can think to do it is to cache the lookup yourself - hold a table with the titles and ids, and write something to accept the title, lookup the id, and call the original title program with the id.

Of course you will then run into disambiguation problems when there's more than one instance of a title.

Actually yes, that's exactly what I'd like to do since I wouldn't know the title id before I do a search for an author and parse the returned page.

Can it be done somehow like the below to directly fetch a title?(Far less fetching and parsing than the way I am doing it now and I know that's not the cgi directive I am using as an example)

http://www.isfdb.org/cgi-bin/title.cgi?the_brethren

(Am also trying to figure out ways of parsing the wikipedia entry to just get the comments for the book but haven't got that far yet (grin))

Could you clarify what you're trying to do please?

It looks like you want to retrieve the ISFDB's page for a particular title in the same way as you have done for the name?

Using a GET rather than a POST, you could use the links as displayed on the John Grisham page you show as an example.

So on that page the first Novel "A Time to Kill (1989)" has a link to http://www.isfdb.org/cgi-bin/title.cgi?183131
and going to that link displays the page for that title.

What extra functionality do you need over that - are you trying to search by the text name rather than the ID number?

Rafe, thanks for your help in the past. I finally decided not to attach the offline database to my code as it was too big for some of my users to handle. Now I am querying and parsing the pages from my code.

Now am trying to figure out what the POST method is for a HTTP query for titles.

If I do:

http://www.isfdb.org/cgi-bin/ea.cgi?john_grisham

I can get the author's page up, no problem there.

But what is the cgi string to retreive a title? Does anyone know please? Thanks all.

fangbeast@gmail.com

Guess I have to do some more research!

Cyril: It's some time since I downloaded the database, and I can't remember all the details. I think it's in UTF-8 format.

I have just downloaded the database onto my Windows machine (sorry, no Unix, nor MySQL) intending to load it into Paradox. I found a lot of (to me) strange character combinations, standing for accented characters, etc. I have laboriously converted them -- except for the '%%' in the Notes file -- but I wondered if there was some standard conversion routine that would work in Windows?

Second question - What should I do about errors which I find?

Cyril

Has anyone got admin rights to delete the link spams, or to add some way to block the posting of them?

The mysql database dump worked nicely, after a minor change to the create table statement for the version I was using (4.0.15 I think) for linux.

Nice work with that!

Hello, perhaps I'm being blind, but I cannot seem to figure out a way to get the numbers of SF books published in a particular year. I'm trying to get this information so I can see how well it correlates with Worldcon attendance since 1950. The quick search by year does give the number of matches and I could use the advanced search and then count the number of results, but both seem to be less than ideal.

Mishalak
at gmail and elsewhere.

Let me introduce a resource that you all may be interested in. But first let try to establish some credibility. I work in the Office of Research of OCLC Online Computer Library Center, inc. You can check us out at : www.oclc.org. Some of you may actually work for us, as there are many Sci Fi readers in our organization. If you go to the site you'll see we are a non profit, you can also check us out on Wikipedia, or Google.

The resource I'm hoping you'll check out is called FictionFinder and can be found at http://fictionfinder.oclc.org . What you see there is the splash page with Alice. go in and try it out. A new interface is about to be installed in the next week or so, so your experience may vary. Also note that this is a prototype system and may or may not be up, although availability has been good.

Note also that we are currently only providing access to books/printed material at this point. Let me know what you think at thompson@oclc.org

Eric,

"The Postman" is definitely not the other book I was thinking of. It could be the one from Nourse, "The Fourth Horseman", although I can't seem to find a synopsis of the book anywhere.

Does the cornucopia of knowledge known as Sally know the synopsis, or where to get it, or by chance have the book? ::grin::

The Barnes and Noble website has a fairly decent browsing function, but it's still somewhat difficult to narrow down to what you're really looking for. I'll see if I can't do a bit more research myself now that I've got a starting point.

Jeff

The Missuz -- or Sally, as she likes to be known -- thinks the other book might be "The Fourth Horseman" by Alan E. Nourse.

"The Postman" might also qualify, if I recall your earlier message correctly. The disease was a bio-weapon, or a bio-weapon made people susceptible to disease, or a bio-weapon made people susceptible to a bio-weapon disease, or ... (Guess you're not the only one who needs to re-read books from years ago.)

Is there a list of post-apocalypse SF books somewhere? There used to be a collection of such lists -- alternate history, time travel, etc. -- but I don't recall where it was or what it was called.

I guess you could give google a list of titles in a category, and see what pages turn up with all of them. At least some of them will be lists of books in that category, right? -Eric

I think that's the book! I believe I might have been mixing up another book's plot with this one, as far as some sort of plague is concerned. Hmmm, wonder what book that was?! ::grin::

In any case, thanks Rick, Eric and the missuz for the help, much appreciated!

Jeff

Here's what my wife found. Could this be it? -Eric

Wolf and Iron by Gordon R. Dickson

Review by School Library Journal
The U. S. has been devastated by worldwide financial collapse. Civilization as readers know it has disappeared. Marauding bands are terrorizing the countryside, killing and looting. Jeremy Bellamy Walthers' goal is to cross 2,000 miles of ravaged countryside to reach the security of his brother's Montana ranch. En route he befriends a wolf who becomes a partner and companion via verbal and nonverbal communication. The story deals with Jeremy's interaction with the wolf and the other human survivors of the economic collapse. Dickson has created another superior novel; it's colorful, well written, and peopled with well-developed, multidimensional characters. The wolf is especially fascinating. YAs who have cut their teeth on such works as George's Julie of the Wolves (Harper, 1972) or Mowatt's Never Cry Wolf (Little, 1963) will enjoy this survival story in sci/fi clothing. --John Lawson, Fairfax County Public Library, VA

Review: Copyright 1990 Cahners Business Information, Inc.

It could be -- if I'm remembering the title right -- "Wolf and Iron". Or something like that. Been a while since I read it, and I don't recall the author. Someone who has been well-known for quite a while, but I'm drawing a blank on who. Probably not Poul Anderson, but someone of that generation, or a bit younger.

I'll check with The Missuz, and see if she remembers it better than I do. Maybe she can give and/or correct the details, so you have something useful to go by.

If you don't get your answer here, you try Asimov's SF message board, here: http://www.asimovs.com/discus/

They do a lot of this kind of thing, and have a couple questions like this going on right now.

Rick, thanks for the help, but unfortunately that's not the book. I took a look at the synopsis, and while similar, it's not the one I'm looking for.

In addition to what I posted about remembering, I also remember a part where the man is travelling through a wooded area and gets attacked by a bear. He has a dog along, and it's the dog that eventually scares off the bear. He had the dog lick the wounds to help prevent infection, which it didn't do. In his travels, the man was bartering for food using a coin collection he had, gold ones, I believe. Barter, or gold, were the only methods available since society, and the monetary system had collapsed.

Maybe those additional details will help jog someone's memory. Strange that I can remember so much of the book, just not the title!

Sounds like it could be George R. Stewart's EARTH ABIDES, published in 1949. That novel took place mostly in California, as I recall, but the plot sounds similar.

I'm looking for a book, which I have read many years ago, that I can no longer remember the title. I would really like to find this book and add it to my library. Hoping someone here can help.

Synopsis: Set in close to the current time, some type of disease spreads across the planet, crippling society. If my memory isn't too far off, I think the disease might have been man-made, and that it might have been spread via the glue on envelopes or postage stamps. In any case, society crumbles as untold #'s of people perish, and technology goes by the wayside. It becomes a world of survive the best you can, any way you can. A lone man makes his way westward and the story tells of his trials and tribulations during his travels.

Unfortunately, that's all I can remember. Hope it's enough!

who is running this site? and i need them to contact me immediately....thanks

free blog's
http://www.4drk.com
[url=http://www.4drk.com]Free blogs![/url]

A short novel by James Blish, "A Case of Conscience" is missing from the listing for Worlds of IF, Sept. 1953

Easiest way to find what they mean by SQL format is probably to export something from it to SQL and look at the result.

Yes, the latest version has been normalised to use links between tables much more, and is much trickier. Far better for the database but harder for people.

I suppose that it might end up being easier to run it through MySQL. (But you'd still need to interpret the links to build the query.)

One thing to watch though - the format that's used to describe tables in backup.gz sets the CHARSET as latin1 but doesn't set the COLLATE to latin1_general_ci. So whatever you've set MySQL to do, the tables get the default collation for latin1 which for some reason is Swedish!

I suspect this is drifting rather off topic - if I can help any more it might be better to take it to direct email. My name at filk co uk

CSV and SQL. But I'm not sure what they mean by SQL format. It could be anything.

I have a very old copy of the IFSDB as text files that someone dumped as some sort of formatted text and wrote a parser for it but it is very old and inaccurate in many cases, that's why I am chasing the latest version.

Because I am no expert, I tried following the indexes from the Backup.gz dump but they didn't make sense to me. The old text dump I had was exported with literal data in each field, instead of link and was immediatley useful to me. Not to mention not giving me a brain ache!!

Kind regards.

fangbeast@gmail.com

What formats will Sqlite read? It might be possible to massage the file - it's plain text.

I was afraid you'd say that. Okay, looks like I have to install MySql server again and learn a few things:):)

The downloadable backup of the database is in the form of an SQL script, so you might be able to massage that into a form that would read into Sqlite?

However it has extremely long lines - many editors won't handle it.

Once it's in the database it should be easy to write a query to create the output tables you need.

Edited by author 12-14-2005 06:22 PM
Is it possible to get a copy of the database in Sqlite 3 form? I was wondering how hard it would be to do and to populate each field with actual data instead of links.

E.g in BOOKS publisher might be a link to the publisher index. I wanted to combine the output to include publisher name instead of link (For a project I am working on)

The database and wiki seem to be all working again.

But looking at the Wiki Recent Changes page shows that it's under heavy attack by link spammers.

I've been getting that error 500 for several days now, every time I try to access the database.

For the last couple of days I've also been getting an error from the ISFDB Wiki:

Fatal error: Call to a member function on a non-object in /www/isfdb.tamu.edu/root/wiki/includes/ObjectCache.php on line 409

Is this about the message I've been getting since Sunday morning? Or is this just me?


  Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, webmaster@academy.tamu.edu and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.

------------------------------------------------------------------------ Apache/2.0.55 (FreeBSD) DAV/2 mod_fastcgi/2.4.2 SVN/1.2.1 mod_ssl/2.0.55 OpenSSL/0.9.8a Server at isfdb.tamu.edu Port 80



QT - grendelkhan wrote:

>
>
>
>
>
< replied-to message removed by QT >

How can you tell? It seems to be stuck on an error 500 for any page I can think of...

I'm sorry to say that the ISFDB database is almost wholly unreliable. In looking at my own entries, there are multiple errors in dates, title information, and publication data--and the vast majority of my books in the field aren't listed at all, although the information is readible ascertainable. I've tried sending in corrections to this material in the past, but I might as well be talking to a blank wall for all the response I get. None of the mistakes are ever fixed. There's even a ghost title listed. Pretty scruffy all around.

Robert Reginald

I have been hoping that the info in the "Top 100 Lists" section will be updated at some stage, as it doesn't appear to have been done for some time. Are there any plans to do this in the near future?

  For Paul Linebarger (and the variants of his name), it says that his name is a pseudonym of Cordwainer Smith. When the truth is that "Cordwainer Smith" is a pseudonym of Paul Linebarger.

  The books listed as non-fiction by Cordwainer Smith were written by Paul Linebarger, as Paul Linebarger, not Cordwainer Smith.

Is everything all right? It's been months since June, Moneths since the travel, and almost 2 months since the last update to the blog.
I understand that you were trying to get this finished so you could 'Unload' most of the work related to the site off to the Fen, but until the last step of opening up the site for editing takes place, we are left with checking back more and more infrequently. Just a Notice or a posting of I'm still here, and I'm still going to open up the database soon would offer us much hope.

Thanks
Kevin

I see a lot more questions than answers, so I don't know how useful this post will be. Here goes...

I hope to use the ISFDB on my own site (with attribution). Thanks for the great work done so far!

I have some PHP which allows me to go from a DB Designer 4 file to a working database in a very short time. I would like to suggest that you post the DB Designer 4 schema file in your download section.

Also, a question: the weak-association tables in the database have a primary key. I can't for the life of me figure out why. I'm sure there's a good reason. I plan to delete them, but I'm curious as to why they are there (I don't plan to use any of the Python CGI code).

The "correct" primary key for a weak-association table, as I understand it, are the two FK's in the M:N relationship.

Again, thanks for making this database available for all of us to use!

A bibliography for the author (not the illustrator) Richard Powers: http://www.richardpowers.net

I have been looking for years for a juvenile sci-fi about a couple of children who stumbled onto some international plot revolving around a weapon that killed by emitting sound at frequencies to destroy buildings, land masses, humans, etc. It was set at a lakeside camp or the like, and involved a boat where the weapon was housed. At one point someone is killed by the weapon, and the children are witnesses, but are unable to get anyone involved to help...

It was a paperback, bookmobile fodder, and I read it in the late 80's. If you have any idea what this might be, please email allegria_joy@yahoo.com . Thanks!

I'm a book dealer, and I've recently come into possession of a few magazine collections. Among them are a full run of Magazine of Horror, a nearly full run of Startling Mystery Stories (missing issue #16) and a number of other issues which aren't listed here.

SMS is most notable for containing the first published stories of both Stephen King and F. Paul Wilson, while Magazine of Horror contained some original fiction from people like Roger Zelazny and Dr. David H. Keller amidst the reprints.

Is there any way I could pass along the contents information before these get sold?

Edited by author 08-16-2005 07:12 PM
Reviewers are calling this novel brilliant, in the tradition of George Orwell, Mark Twain, and Kurt Vonnegut. Destined to become a cult classic of speculative fiction.

http://www.waronjunk.com (turn your speakers on)

On the ISFDB2 Status and Documentation page there is a link to the download a SQL database backup, but it looks like the file has been removed from the site. Would it possible to repost the file?

I was just looking at Fred Pohl's listing and noticed that his new book, THE BOY WHO WOULD LIVE FOREVER, is not listed under the Gateway series. Out of curiosity I used the editing tool, even though it's not active, to see if there was a "series" tag, and there wasn't. Will this be available in the new finished product?

ISFDB seems to have an error. There are two authors listed as having written a short story named "Integration Module" in 1973: Donald B. James (in "Analog") and Daniel B. James (anthologized in "Sociology Through Science Fiction", 1974, St. Martin's).
http://www.isfdb.org/cgi-bin/title.cgi?51113
http://www.isfdb.org/cgi-bin/title.cgi?98096


I think one of the two entries is in error, and there's only one story.

Who do I report this to? Is there anything I need to do first? -Eric

Personally, I've been holding off on some personal catloging so I can update the ISFDB at the same time. (It's always nice to have the physical article in hand as opposed to back updating from a static catlog)

> Is there anyone particularly interested in magazines and
> journals. I see that we have *some* in the database.

Well, my schedule is as unpredictable as ever, but I should be
able to fill in a number gaps once the editing tools go live.
1930s-1960s magazines will be my first target.

> I will be pushing for as much data as possible to be submitted
> to the ISFDB too.

At this point general cleanup and data reorganization are probably as important as entering new data, but there is no reason why they can't be going on in parallel.

--
Ahasuerus


--
___________________________________________________________
Sign-up for Ads Free at Mail.com
http://promo.mail.com/adsfreejump.htm

Ah! After a bit more poking around, I noticed that the content editing page at [ http://www.isfdb.org/cgi-bin/edit/editcontent.cgi?3239 ], for instance, uses raw entry of publication dates and titles, instead of title references. That *would* involve a lot of work to change, but it seems to me like it really should be done with references.

Or maybe that whole page is out of date and that's why the link to it was intentionally broken. It's clearly a work in progress---there's an extra form open tag in there, even.

A suggestion: in editpub.cgi, clarify that "Price" signifies the cover price, not the current going price. Make Pub Format a pulldown with friendly labels ("Paperback, Hardcover" instead of "pb, hc").

Also, the "new publication" (editpub.cgi?0) form doesn't have a field for the title link; how does that get put in? Is it created by going from title.cgi?xxx to the 'new book' link?

The link at the bottom of editpub.cgi links to /cgi-bin/editcontent.cgi; it should link to /cgi-bin/edit/editcontent.cgi. Or editcontent.cgi needs to be moved, either way.

And I think publication SSGNRNTY191960 is mangled.

Question: what's the canonically right way to enter, for instance, publications for title 25893 (Heinlein's "Assignment in Eternity", a short story collection reissued several times)? Publication SSGNMNTETR1953 apparently contains c (Collection?) "Assignment in Eternity" as well as two novellas and two novelettes. (What are the criteria for novels, novellas and novelettes?) On the other hand, publication SSGNRNTY1987 contains only a single c entry. Presumably this entry leads back to title 25893, so either the system is redundant for including that 'c' entry, or there's no way to (correctly) get an actual list of the contents of the collection. I'm a bit confused here.

Feature request: on the Publication Listing, can short works link to their own title pages? For instance, [ http://www.isfdb.org/cgi-bin/pl.cgi?SSGNMNTETR1953 ] should have a link to [ http://www.isfdb.org/cgi-bin/title.cgi?41342 ] in some fashion. Of course, title 41342 would need to be somehow linked to titles 113040 and 113041, wouldn't it?

Journals

Is there anyone particularly interested in magazines and journals. I see that we have *some* in the database. I am talking to people in the UK about entering the entire run of a particular journal into a database. It would make sense if we used the ISFDB format so as to not re-invent the wheel.

I will be pushing for as much data as possible to be submitted to the ISFDB too.

There are two ways to merge titles. If the titles are on the same bibliography then click on the [Titles] link on the author bibliography, and check the merge box of the two titles you wish to merge, then click on the "Merge Selected Records" button.

If the titles aren't on the same page due to the number of works, then go to Advance Search and search directly for the titles via a Title Search. Then follow the above steps.

Will there be an interface to merge title records? I see one to unmerge them, but it seems that a far more pressing question is how to deal with duplicates such as 1863 and 28815? (Incidentally, both Amazon and Wikipedia use the unhyphenated name.)

I'm attempting to familiarize myself with the interface in preparation for the site going live. Where will the record merging be?

Interestingly, this listing shows Zelazny's Amber series appearing twice and what appears to be an anthology series mixed in with the novels.
    * Amber
          o Roger Zelazny's Visual Guide to Castle Amber (1988) with Neil Randall

    * Author's Choice Monthly
          o 27 Author's Choice Monthly, Issue 27: Gone to Earth (1991)

    * Bring Me the Head of Prince Charming
          o Bring Me the Head of Prince Charming (1991) with Robert Sheckley
          o If at Faust You Don't Succeed (1993) with Robert Sheckley
          o A Farce to be Reckoned With (1995) with Robert Sheckley

    * Francis Sandow
          o Isle of the Dead (1969)
          o To Die in Italbar (1973)

    * Amber
          o Merlin
                + Trumps of Doom (1985)
                + Blood of Amber (1986)
                + Sign of Chaos (1987)
                + Knight of Shadows (1989)
                + Prince of Chaos (1991)
          o The Chronicles of Amber Pentalogy
                + Nine Princes in Amber (1970)
- - - - - - - - - - - - - - - -

Thanks for the response.
A further suggestion for the upcoming titles section...

When the list of upcoming titles gets to less than 3-10 days left in the month, The system should start displaying the first 3-10 days of upcoming releases next month.

Rational: A once a month or a once a week visitor, who visited the site last time, just before then end of the month, and then next time after the first tuesday of the month will miss seeing a notice of an upcoming release that is tied to the standard media scheduling at the beginning of the month.

Thanks

BTW, Catherine Crook de Camp (http://www.isfdb.org/cgi-bin/ea.cgi?Catherine%20Crook%20de%20Camp) is aka Catherine A. Crook (http://www.isfdb.org/cgi-bin/ea.cgi?Catherine%20A.%20Crook). Both pages have a fair number of entries, so her bibliography is effectively split in two.

That's right, the "^" character is (was?) supposed to be used to separate alternate titles during data entry, but I don't think it is supposed to be displayed on biblio pages :)

Ahasuerus: I think the ^ characters are variable titles. See http://www.isfdb.org/cgi-bin/pl.cgi?SCNDVRTY1987 , where one story in the collection is listed as

Impostor^Imposter, ss, (1953)

That title is more accurately listed in http://www.isfdb.org/cgi-bin/title.cgi?55911 as Impostor [vt Imposter]. Curiously, that page links to the above collection, but the above collection doesn't link to the short-story title.

What a peculiar book!

http://www.isfdb.org/cgi-bin/title.cgi?30452 :

Bibliography: Sci Fic Sep-20mxctn: Mon-16ctd

Title: Sci Fic Sep-20mxctn: Mon-16ctd
Authors: Et Al Card and Poul Anderson
Year: 1992
Publications:

Sci Fic Sep-20mxctn: Mon-16ctd (1992 , Tor, 0812529464, $91.64, 544pp, tp)

Something appears to be not quite right with alternate title display logic for omnibuses, e.g. http://www.isfdb.org/cgi-bin/pl.cgi?FLANDRY1993 :

NOVEL A Circus of Hells Poul Anderson
NOVEL The Rebel Worlds^Commander Flandry (1978 UK) Poul Anderson

Ahem:

==================
http://www.isfdb.org/cgi-bin/eas.cgi?Robert%20A.%20Heinlein

Robert A. Heinlein - Summary Bibliography (Short Works)

Pseudonym For: Frederik Pohl , Judith Merril

No short works found for Robert A. Heinlein
==================

Who knew?!

The Wiki page and the main ISFDB page both appear to be down :(

1] The ISFDB database has been successfully updated, and is now sane again.

2] Jeff Bachtel has brought the ISFDB Wiki online at http://isfdb.tamu.edu/wiki/index.php/Main_Page . I've populated a few high level areas into the main page. I'm going to focus on the Design Documentation area myself, so if anyone wants to pitch in and start moving static ISFDB pages into the Wiki, feel free (with the exception of the FAQ - the current FAQ is too old and out of date to bring over - we really need to start fresh). I also put in Reported Bugs and Features sections; if I've missed any feel free to add to them.

Al von Ruff thanx for reply
 quote
Quietsilence> I haven't put up a new downloadable version because I'm behind on updating the documentation, and the database layout is so normalized that it probably requires some explanation. The problem going down the road is that we'll have information in some tables that should not be redistributed (such as user email addresses) that will be required for future features. It would be interesting to know what the requirements are for a downloadable version of the database - perhaps we can generate something specific to that purpose.

I use the down loadable database to sort and rename ebooks
the app looks at the Isfdb to get info like author name, title, and series names also to see if it's a shortstory collection or not
 if your interested in looking at the app you can find it here http://www.geocities.com/vangelis_df/vBookSorter/vBookSorter.html

its not my app but is wonderful for renaming ebooks it works standalone but is much better with the ifsdb

AvR>>(I say almost, because there are stories of obvious professional quality being e-published on websites).<<

There is a growing amount of uncopyrighted pre-1923 SF on the Web. Give Google (and its digital library team) a couple of years and there will be more, much more. Do we want to link them all? Given enough time and a sufficiently stable URL base, perhaps...

AvR>>Dublin Core, there are 3 things to track: works, expressions, and manifestations.<<

Interesting, need some time to chew on it.

AvR>>I think it falls into two categories: metafictional and author-centric.<<

That sounds like a very good summary. The only problem with metafictional data is that it can get unwieldy to track down and catalog. There are too many fanzines out there and, besides, there may be parallel efforts to catalog them. Need to look into that.

AvR>>Author-centric would be completely non-associational nonfiction which is present to complete the bibliography of an author predominately known for producing fiction.<<

That sounds very reasonable, except that I would change "an author predominately known for producing fiction" to "an author predominately known for producing speculative fiction". For certain values of speculative fiction, of course.

AvR>>we want textual items that adhere to rule #1 and rule #2, AND have passed some kind of bar for publication.... As always, there are author-centric loopholes - if Robert Heinlein had a self-published book we would obviously list it.<<

My first instinct was to suggest that we should include any non-profesionally published works by people who are also established professionally published authors. However, many professionally published genre authors continue publishing in amateur publications long after they become professionally successful: Lovecraft, Asimov, etc. That would bring us back to the fanzine field, which, like manga, is not a can of worms that I would be comfortable opening until and unless we have a better understanding of what we are getting into.

AvR>>Seems to me that we're still at it 10 years later...<<

Oh no, I would say we are up to at least "63% of the way decent design"!

>>1] "Ace", "Ace Books", "Ace Science Fiction", "Ace Fantasy", etc...<<

Sure, but that's really imprint information. It's just that the imprint is sometimes the same as the name of the publishing house or at least fairly close to it. I would argue that we would benefit from capturing both the imprint and the publisher, when available. In many cases deriving the publisher based on the imprint (Orb => Tor) should be easy as well.

Another random observation: Common series names.

Guess what happens when the name of a series is very common, e.g. "Wizard"? Yup, entries from more than one series show up together when you clik on the link: http://www.isfdb.org/cgi-bin/pe.cgi?Wizard . Worse, the superseries ("universe"? I forget the agreed upon terminology) name is also "Wizard" in Hawke's case, which makes things even more confusing.

Is there a benefit to disambiguating series/superseries names? Should we put some rules of thumb in place, e.g. "no superseries name can contain a series whose name is the same as that of the superseries"? Should we encourage people to append respective authors' names to the end of series name when applicable, e.g. "Wizard (Simon Hawke)" vs. "Wizard (Diane Duane)"? It won't work for shared worlds, but hey, it's a start!

Ahasuerus> just think how long it took us to come up with a half-way decent database design in 1995 even though we already knew a few things about the field.

Seems to me that we're still at it 10 years later...

Ahasuerus> Any way to hyperlink publisher names

Yeah, but. 2 problems:

1] "Ace", "Ace Books", "Ace Science Fiction", "Ace Fantasy", etc...

2] Hundreds (sometimes thousands) of results.

Eventually this will replace the static publishers lists, but I still haven't thought through the details.

Al von Ruff>>The other thing I'd like to do is put a verification field in a publication that can be signed when someone compares the entry to the ACTUAL publication - like they-have-to-hold-it-in-their-hands kind of verification.<<

Yes, that would be helpful, especially when it comes to vaporware publications like "Elves and Muskets" (http://www.isfdb.org/cgi-bin/title.cgi?5829). However, you have to be careful not to confuse the baby with the bathwater. For example, one of Frank Belknap Long's stories in _Unknown_ was not included in the table of contents, which can confuse the unsuspecting. But then again, nobody ever said that SF bibliography was for the faint of heart!

As far as manga goes, I am not sure the ISFDB would be a good place to catalog it. For starters, we may not have the requisite knowledge to do the field justice: just think how long it took us to come up with a half-way decent database design in 1995 even though we already knew a few things about the field.

P.S. Any way to hyperlink publisher names, e.g. "Ace", to their respective pages?

Ahasuerus> [BTW, Al, feel free to tell me when to stop spamming the board. I have an unhealthy amount of free time on my hands right now. One heck of change.]

I'm burning time tonight myself while I'm waiting for the database to return to sanity.

> Forthcoming books...

I do try to filter the crap, but there's just too much to do right now, so stuff like calendars are getting through. The Manga and comic books are interesting in that much of it has been categorized as science fiction. So do we list Science Fiction Manga?

There's also a crapload of children's books that show up as well.

> Checklists.

Awesome idea. I've done that in the past with offline lists, but putting them in the database itself could be helpful. The other thing I'd like to do is put a verification field in a publication that can be signed when someone compares the entry to the ACTUAL publication - like they-have-to-hold-it-in-their-hands kind of verification.

Ahasuerus> What's a "publication"?

On one end of the range there is everything (well almost everything) that has an ISBN; these obviously qualify for inclusion (I say almost, in that there are items such as calendars, notepads, and associational statuettes that should obviously not be included).

On the other end of the range there is everything (well almost everything) that is a webpage; these obviously should not be included, otherwise we might as well hand everything over to google. (I say almost, because there are stories of obvious professional quality being e-published on websites).

Okay, so even the endpoints aren't cut and dried. Let's back up a bit and talk about how Dublin Core categorizes things. In Dublin Core, there are 3 things to track: works, expressions, and manifestations. A Dublin Core work maps roughly onto ISFDB titles, and a Dublin Core manifestation maps roughly onto ISFDB publications. Starting with a text, it might be expressed as a short story, a chapter book, an illustrated child's book, an audio book, or a graphic novel. Each of those expressions could have multiple manifestations (or publications). So:

1] To use Dublin Core terminology, we're interested in textual works - not movies, or radio shows, or paintings. The textual work might later be expressed as a movie or radio show (and that information may be of secondary interest, but it's not our primary focus), and a radio show or movie might later be made into a textual work (like the Hitchhicker's Guide to the Galaxy), but our focus remains textual works.

2] In general we're predominately interested in textual works of fiction, but we'll tolerate a good amount of nonfiction. So how much? I think it falls into two categories: metafictional and author-centric. Metafictional works would be nonfiction works about fiction - interviews, reviews, general essays on fiction, or publishing, or conventions, and so on. Author-centric would be completely non-associational nonfiction which is present to complete the bibliography of an author predominately known for producing fiction. We should not stray into academic papers, for instance (a rule which, of course, we violate in David Brin's bibliography - see author-centric). We don't need to list all of the Idiot's Guides for Dummies (although I guess a copy of "Bibliographic Databases for Dummies" would be okay).

3] This one's a bit thornier - we want textual items that adhere to rule #1 and rule #2, AND have passed some kind of bar for publication. That is, some editor has evaluated the work and has deemed it suitable in some way for publication (whatever that might mean). This eliminates most web pages (but not SciFiction), and eliminates vanity presses and self-published books, and may include amateur publications and e-books based on the situation. As always, there are author-centric loopholes - if Robert Heinlein had a self-published book we would obviously list it. However, we shouldn't list someone simply because they self published 10,000 lines of "All work and no play makes Jack a dull boy."

So let's start with those 3 rules and poke holes in them or elaborate on them.

[BTW, Al, feel free to tell me when to stop spamming the board. I have an unhealthy amount of free time on my hands right now. One heck of change.]

Reviewing the list of "forthcoming books", I see all kinds of comics, manga, calendars, even RPG rule books (!) Is that by design or just an artifact of wholesale importing from Amazon.com?

Also, there are a lot of duplicate entries, messed up titles, incorrectly attributed authors/artists, etc. Is there a flag that could be set on a record by record basis for "raw input"? That way we could generate reports of "raw" records, correct them, and then flip the flag once done.

Finally, it might be helpful to have a list of checkmarks for individual authors: checked against Bleiler, checked against Reginald, checked against the LoC, checked against Gaslight, Contento, Hall, The SF Hub, Bookwhere, etc.

Back to basics.

What's a "publication"? Professionally produced paper books (including small presses) obviously qualify and so do audio books. But:

1. Vanity presses? If not, how do we define "vanity"? (This is not always an easy question to answer).

2. Self-published books? (Only if they were later reprinted by major houses a la Halperin?)

3. Amateur publications, e.g. Lin Carter's early poetry? Fanzines?

4. E-books? CDs? Web pages?

On a lighter notes, there may be value to including a pointer to "related short works" when listing series data under "long works".

For example, Bujold's Miles series includes a couple of shorter entries, "The Mountains of Mourning" (1989) and "Labyrinth" (1989). Currently, they are not listed as part of the saga, but even if they were, they would be invisible on the main ("long works") page and there would be no indication that there was more to the series than met the eye. Similarly, the main page for Neil R. Smith gives no indication that there were more Professor Jameson stories than what was collected in the 5 reprint collections.

There was a problem in uploading a new version of the database today, and one of the tables (pub_content) is truncated; which is why you can't see any publication contents.

The database will be restored to its original luster in a couple of hours, and an up-to-date version will be put up by tomorrow.

Ahasuerus> Secondary bibliography.

Not currently. From a database table layout point of view, it's a one line change (maybe a no line change). From an application point of view, they could easily be collected and displayed on the main summary page or in a separate secondary bibliography.

The real change needed is how to get the data into the table. Works of nonfiction and essays would need an extra line to designate a subject author (similar to interviews and reviews, both of which we have finished tackling the editing forms yet).

Bottom line - not supported; could be easily supported; need to think through UI issues to make sure field isn't abused or confusing.

PortForlorn> ..."Anthology Series" section now appears near the end of the long works...

Here's the old, isfdb1 order (actual code):

        PrintSeries(bib, SERIES_TYPE_OTHER)
        PrintEntries(bib, 'n', 'Novels')
        PrintEntries(bib, 'c', 'Collections')
        PrintEntries(bib, 'o', 'Omnibus')
        PrintSeries(bib, SERIES_TYPE_ANTH)
        PrintEntries(bib, 'a', 'Anthologies')
        PrintEntries(bib, 'e', 'Magazine Editor')
        PrintSeries(bib, SERIES_TYPE_NONFIC)
        PrintEntries(bib, 'nf', 'Non-Fiction')
        PrintEntries(bib, 'sf', 'Short Fiction')
        PrintEntries(bib, 'p', 'Poems')
        PrintEntries(bib, 'ng', 'Non-Genre')
        PrintEntries(bib, 'ar', 'Cover Artwork')
        PrintEntries(bib, 'ai', 'Interior Artwork')
        PrintSeries(bib, SERIES_TYPE_ESSAY)
        PrintEntries(bib, 'es', 'Essays/Articles')
        PrintEntries(bib, 'r', 'Reviews')
        PrintEntries(bib, 'i', 'Interviews')

If we take out the short works, the order was:

        PrintSeries(bib, SERIES_TYPE_OTHER)
        PrintEntries(bib, 'n', 'Novels')
        PrintEntries(bib, 'c', 'Collections')
        PrintEntries(bib, 'o', 'Omnibus')
        PrintSeries(bib, SERIES_TYPE_ANTH)
        PrintEntries(bib, 'a', 'Anthologies')
        PrintEntries(bib, 'e', 'Magazine Editor')
        PrintSeries(bib, SERIES_TYPE_NONFIC)
        PrintEntries(bib, 'nf', 'Non-Fiction')
        PrintEntries(bib, 'ng', 'Non-Genre')
        PrintEntries(bib, 'ar', 'Cover Artwork')
        PrintEntries(bib, 'ai', 'Interior Artwork')

The ISFDB2 order is:

        bib.printNCSeries()
        bib.displayWorks('NOVEL')
        bib.displayWorks('COLLECTION')
        bib.displayWorks('OMNIBUS')
        bib.displayWorks('SERIAL') <===
        bib.displayWorks('EDITOR') <===
        bib.loadASeriesData()
        bib.printASeries()
        bib.displayWorks('ANTHOLOGY')
        bib.loadNFSeriesData()
        bib.printNFSeries()
        bib.displayWorks('NONFICTION')
        bib.displayWorks('NONGENRE')
        bib.displayWorks('COVERART')
        bib.displayWorks('BACKCOVERART')
        bib.displayWorks('INTERIORART')

In ISFDB1, any orphaned serials where printed at the end of the 1st PrintEntries() (which would have placed them after the Novels section).

So the two differences are:

1] Orphaned serials are now printed after the omnibus section instead of after the novel section (but they were always ahead of the anthologies, either in ISFDB1 or ISFDB2).

2] The editor section sneaked ahead of the anthology section. Since Weber hasn't edited any magazines, the order of his bibliography should be roughly the same.

So just to check to make sure the order didn't change 2 years ago when I converted from C to Python, the order of the C-based app was (with short works removed):

        print_series( title_list, exact_author, 0, 0 );
        print_entries( title_list, exact_author, "n", "Novels", 0 );
        print_entries( title_list, exact_author, "c", "Collections", 0 );
        print_entries( title_list, exact_author, "o", "Omnibus", 0 );
        print_series( title_list, exact_author, 0, 1 );
        print_entries( title_list, exact_author, "a", "Anthologies", 0 );
        print_series( title_list, exact_author, 0, 2 );
        print_entries( title_list, exact_author, "e", "Magazine Editor", 0 );
        print_entries( title_list, exact_author, "nf", "Non-Fiction", 0 );
        print_entries( title_list, exact_author, "ng", "Non-Genre", 0 );
        print_entries( title_list, exact_author, "ar", "Cover Artwork", 0 );
        print_entries( title_list, exact_author, "ai", "Interior Artwork", 0 );
        print_entries( title_list, exact_author, "se", "Serials", 0 );

1] Here orphaned serials were dead last.
2] Again anthologies were listed before the editor.

The old C-based ISFDB intermingled long/short titles in the first series listing, which the new ISFDB no longer does, so in some respects, the anthology listing has moved up a tad higher from those days.

I'm pretty sure that I've always tried to group the concept of "anthologies" together (anthology series, followed by loose anthologies), rather than try to group the concept of series together.

Secondary bibliography.

Is there a way to link bibliographical works to the people that they cover? For example, if I were to take the Simak (bio-)bibiliographies that I listed in his Wikipedia entry and enter them in the ISFDB, would the users be able to see them when they pull up Simak's bibliography?

Re: anthology series. Sorry to have been unclear. What I meant to say was that "Anthology Series" section now appears near the end of the long works listing. (For example, see David Weber.) It is my recollection that they used to appear up front with the other series titles and before the listing of non-series titles. I was wondering if moving that section of the long works listing was deliberate. I'd personally prefer to see all the series related publications together but I suppose I could get used to the new order of sections in the long works listing {if forced to ;-) }.

The full title is "The Grand Tour: Being a Revelation of Matters of High Confidentiality and Greatest Importance, Including Extracts from the Intimate Diary of a Noblewoman and the Sworn Testimony of a Lady of Quality".

And it's a good thing that you used "MEDIUMTEXT" or else we would be in trouble with some Victorian titles :)

Ahasuerus> The title field appears to be limited to 160 characters

The field is a MySQL MEDIUMTEXT field, which supports a length out to.. about 16MB. I attempted to edit some bogus text onto the end of the errant record (on the home version of the ISFDB), and it worked fine. It was probably truncated in one of the conversion steps. What's the real title? I'll fix it before we forget.

Ahasuerus> BTW, ISFDB-2 doesn't seem to like apostrophes much,

There are 4 entries in the database (Madeleine L'Engle one of them) in which the apostrophe was converted to its HTML encoding ("’") by mistake (one of the tools in the editing tool chain). I've fixed Madeleine online by hand, but if you do an ISFDB search for "rsquo" you'll see the other 3; I'll fix them soon.

A minor detail: The title field appears to be limited to 160 characters, which works fine for most entries, but look what it does to http://www.isfdb.org/cgi-bin/title.cgi?152790 :

The Grand Tour: Being a Revelation of Matters of High Confidentiality and Greatest Importance, Including Extracts from the Intimate Diary of a Noblewoman and the

Possible solutions:

1. Bump it up to 300 (?) characters.

2. Introduce a free text (non-searchable?) subtitle field.

Kevin P> The ability to sort forthcoming books on a variety of columns is certainly a possibility. Since there's so much data now, it also needs to be filterable. I'll put it on the list of things to do.

Quietsilence> I haven't put up a new downloadable version because I'm behind on updating the documentation, and the database layout is so normalized that it probably requires some explanation. The problem going down the road is that we'll have information in some tables that should not be redistributed (such as user email addresses) that will be required for future features. It would be interesting to know what the requirements are for a downloadable version of the database - perhaps we can generate something specific to that purpose.

PortForlorn> re anthology series no longer being display: Do you have an example? A quick look at Gardner Dozois shows them present. Omnibus series: I think long works should be accumulated in the same list; in particular I'm thinking of works that are published only as an omnibus in one country, and as multiple works in other countries. Could lead to confusion to separate them. Translators, narrators, illustrators: Translators are a little different (since they are essentially creating a derived work from the original), but in general it should work like this: The work should list the author (since he/she created the work). The illustrators and narrators should be listed in the publication (but not the work, since they didn't contribute to that). Illustrators and narrators can differ from publication to publication, but the author doesn't. I'm thinking of modifying publication records such that there is an author, and any number of contributors (annotated by role), so that interesting items like editor, voice narrator, illustrators could be tracked, and perhaps the translator could move over there as well.

A lot of these detailed questions need to go into a new FAQ. As far as moderators go, my rough plan is:

1] Get login facility running (much progress was made this weekend).
2] Give out logins to a select few.
3] Let those people do some edits and gain familiarity with the system, and shake out any bugs.
4] Promote those interested into moderators.
5] Open up the system to everone else, allowing edits without login if so desired.

Target date for enabling editing is July 1.

I love the new "Forthcoming Books" function. Any chance that the list displayed for each month could be sortable by Author and Release Type (HC, TP, MM).
A further nice ability would be sorting new releases by Publisher, by Newness (Original, Re-Release in a new format, etc)

I'm also looking forward very much to the ability to submit updates to the database. Thanks for working on making the capability available.

new sight looks good but i have 1 small prob i cant find the Database to download any more is it still avalable?

PortForlorn>>I notice that you've added a new type designator, "collection". How is this different from omnibus and anthology? Also, will there be (or is there) a "collection series"?<<

Ordinarily, the term "collection" refers to single author collections. Introductions/afterwords by other hands don't disqualify "works" from being considered "collections" and neither do collaborations.

An "anthology" is a collection of short works by different authors. Some people think that it has to contain works by at least three different authors to qualify, so Ace Doubles wouldn't count.

An "omnibus" is normally a compendium of previously published long works, sometimes with short works thrown in for full measure. Back when the original ISFDB design was being thrashed out, I thought that omnibuses would be listed in the "publication" section, but there is a good case to be made for putting them in the "works" section instead. People who use the database probably care more about being able to find and identify separate titles than they care about nomenclatural purism.

PF>>Should an omnibus of titles that were all part of a series be listed as an "omnibus series"?<<

I believe some omnibuses containing serial titles are currently listed as part of that series, but in other cases they are listed separately.

PF>>And how would you like to see an omnibus (or anthology or collection) cataloged that has some titles from a series and some titles that are not part of that series?<<

That can get hairy. Again, technically, omnibuses are publication artifacts and don't fit neatly in the "works" world, but putting them on the main page does make life easier. There is a price to pay, though...

PF>>Should a translator be listed as a co-author or is there a better way to show the translator's participation?<<

There is a separate "translator" field in the new edit forms, I believe.

>>Should a translated title be shown as a new title or as an alternate to the original title?<<

As Al indicated below, the current standard - for a certain value of the term "standard" anyway :) - is to list it as an alternate title. However, if you list all known foreign language titles for something as popular as _The Lord of the Rings_, it can make the page very cluttered.

PF>>How would you like to see catalogued a novel by one author with a foreword by another?<<

Forewords are publication-specific level data. A single novel can have a number of editions ("publications") associated with it, each with a different foreword by a different person. This structure is currently supported by the ISFDB, although the transition period may be making it messier than it used to be.

BTW, ISFDB-2 doesn't seem to like apostrophes much. For example, http://www.isfdb.org/cgi-bin/ea.cgi?Madeleine%20L’Engle yields "Author not found: Madeleine L’Engle". The "Directory" link to her bibliography is http://www.isfdb.org/cgi-bin/ea.cgi?Madeleine_L\'Engle , which yields another "Author not found: Madeleine L\\\'Engle" error.

I'm glad to see that the ISFDB is re-emerging and {gasp!} with an edit capability. I have used and enjoyed it for a number of years and really appreciate all the effort that has gone into creating this database. Thanks, Al!!

I am anxiously awaiting the opportunity to suggest some corrections to the data you have assembled. Over the years I've been mildly irritated to see things like six copies of a title listed correctly as different publication issues of the same book but one or two instances of folks submitting the same title as a new novel. In fact, the "mild irritation" has accumulated to the point where I've promised myself that I'll submit as many corrections as I can find.

I do have a few questions about the new ISFDB though.

I notice that you've added a new type designator, "collection". How is this different from omnibus and anthology? Also, will there be (or is there) a "collection series"?

I also noticed that "anthology series" is no longer being listed at the front with the other series titles. Is this deliberate?

Should an omnibus of titles that were all part of a series be listed as an "omnibus series"?

And how would you like to see an omnibus (or anthology or collection) cataloged that has some titles from a series and some titles that are not part of that series?

Should a translator be listed as a co-author or is there a better way to show the translator's participation?

Should a translated title be shown as a new title or as an alternate to the original title?

Is there a way to designate an illustrator who has provided interior artwork but who is not the cover artist?

How would you like to see catalogued a novel by one author with a foreword by another? {Should a foreword by say Robert Heinlein be noted at all? If the book does contains something written by RAH, the DB probably should indicate that. But they really aren't both "authors" of this novel so showing RAH as a co-author seems wrong. On the other hand, the book isn't really an anthology either.}

Will there be a way to reclassify a title that was entered as a new novel as another publication issue of an existing title?

Will there be a field on the edit form that allows for re-classifying an existing stand-alone title as part of a series?

Will there be a way to indicate that a title is part of a new series?

Will there be a place to deposit scanned copies of cover art for those volumes that aren't available through Amazon?

My last question is on a somewhat different topic. How does one get to be a moderator? {Is this a preternatural gift one is born with? Or does one just volunteer? Is there an internship or qualification test? And is there a study guide for that test?} To paraphrase, what can I volunteer to help with?

AvR>> Since defining either speculative fiction works or speculative fiction authors can be a thorny process, I don't see any reason to form any strict regulations on inclusion. In fact there are already Westerns, Romances, Mysteries, Techno-Thrillers, and even... mainstream works already in the ISFDB. Stephen King fans want to know about all of King's works - even works which are arguable non-SF, <<

Including King's or Stephenson's non-SF works is all good and well, but there are many authors who have committed an occasional foray into speculative areas and yet the bulk of their work is presumably not of interest to the target audience. Take Henry James, for example. _The Turn of the Screw_ is arguably an SF piece, yet the rest of his output was quite mundane. Take Kipling - a short book worth of SF and the rest is mainstream. Dickens - an SF novelette, at best, and the rest is mainstream. Keeler - one famous SF story and dozens of mysteries. The list goes on and on.

Would it make sense to designate authors as "primarily speculative fiction writers" which merit the inclusion of their complete oeuvre or, alternatively, "primarily non-SF writers", whose SF works (and only SF works) should be cataloged? It sounds awkward, but it's best I have been able to come up with so far.

AvR>>So long as the ability to assign tags or keywords to works is available, and users can filter results based on their preferences, I don't see any reason (other than explosive growth) to not open up the ISFDB to fiction in general.<<

It should be noted that there is a great deal of non-fiction that is of interest to SF readers as well. Biographies, primary and secondary bibliographies, memoirs, reviews, etc. The ISFB has quite a few of them already and, if the database is opened up to all kinds of fiction, what's to prevent non-fiction from being added? Starting with Benford's, Forward's and Brin's physics papers :)

AvR>> I think the fact that the database is still called a Speculative Fiction Database will tend to reign in what gets entered, without an explicit non-inclusion policy.<<

Anything that can be abused will be abused. Anything that can't be abused... well, we'll have to see about that :)

>>For authors whose first publications are English, I think things are pretty straightforward: the title of the work is in English, and the publications of that work are listed in their translated language, with the annotation (Latvian Edition), or whatever.<<

That seems reasonable as long as the extra volume is unlikely to present technical problems. How scalable is MySQL?

AvR>>For authors whose publish in a foreign language first (say Stanislaw Lem), the issue is thornier. The correct way would be to list the title in it's original language and add a variant title in the reader's language.<<

But what about, say, German language editions of Lem's books? Shoudl they be included in the ISFDB?

AvR>>This has been done in the ISFDB to a small degree (the best example would be the bibliography for Jules Verne (http://www.isfdb.org/cgi-bin/ea.cgi?Jules%20Verne), in particular the Voyages Extraordinaires is done this way, although others of his books have not.<<

Actually, the verne example brings up another problem. The Voyages Extraordinaires were a "publication" series, not a "works" series. There are a few "works" series hidden in that list and I think it's safe to assume that from the readers' perspective that is more useful information than knowing that all these books were published in the same series. There are a few other examples of these two types of series colliding as well. Any idea wrt how this could be addressed? (Hey, nobody said this would be easy! :) )

Thanks for the pointer grendelkhan; I've fixed that code generator, and things look better. MySQL still doesn't match search strings properly with utf-8 (Mieville doesn't match on Miéville), although it works fine at home with latin1. (not to mention problems I had with Python trying to do MySQL updates with utf-8).

I did make 4 age scripts (oldest.cgi, oldest_nl.cgi, youngest.cgi, and youngest_nl.cgi), but there are numerous date fixes that I've done since the last update, so I need a new copy put online before I release those scripts.

The reason the change of a meta tag doesn't cause the page to render as UTF-8 is that the HTTP header is sending ISO-8859-1. Check the W3C validator, which reports a mismatch between the HTTP header and the META tag. So the META tag is just fine; your webserver is sending the wrong header.

I'll answer some of these tonight, the rest tomorrow:

grendelkhan> all the links point to localhost.

That one's fixed and online (difference in setup between home system and TAMU).

grendelkhan> (Re: oldest lists)

Those are old. I'll put some dynamic script replacements up tomorrow.

grendelkhan> Stranger in a Strange Land[/a][/i]

Fixed that earlier in the day (bad else case dealing with variant titles).

grendelkhan> ...duplicates...

I'll look at those specific ones, but there are others...

grendelkhan> ...UTF-8 stuff...

The version of the MySQL at home supports latin-1, which is what the tables are defined as, but TAMU's running an older version which only supports UTF-8. I've had trouble getting the unicode to match up between browser -> python -> python cgi -> MySQLdb module -> MySQL when using UTF-8, and a lot of success with latin1.

I made some changes to the meta tag to issue the charset utf-8 tag, but it doesn't seem to help in my browser. Do you have a string that works?

Edited by author 06-11-2005 04:48 PM
I posted this on the blogspot message board, but this appears to be the place for it.

Congratulations on the new version! I'm adding some Wikipedia links back to the database; I have to say, it looks mighty spiffy.

On to the nitpicking.

This page (the Editing Tools-->Titles one), all the links point to localhost. I think this page is part of the functionality that's not yet live, but I thought you should know about it.

On Oldest Non-Living Authors/Editors/Artists, there's a '101' at the bottom of the list, which should be at the top. Something's amiss with the sorting. I have no idea what's up with Youngest Non-Living Authors/Editors/Artists. Are these old pages, generated from the old system, and no longer relevant?

Titles 168797, 169060 and 169135 and possibly but not definitely 168780 are duplicates of each other. Also, 172897 and 173209; 172996 and 173362; and 172227 and 172550. Also, Pier Xanthony can most likely be merged as an alias or pseudonym of Piers Anthony.

Title pages are displaying markup in the title. The HTML reads (mangled to get through Blogger's filters)

[title]Bibliography: Stranger in a Strange Land[/a][/i] [/title]

Now, suggestions.

It might be useful to provide Wikipedia links for titles as well as for authors. Wikipedia writes them, anyway.

ISBNs can be autolinked to Wikipedia, which has a really nice ISBN tool. For instance, ISBN 1892391201 (a paperback version of Her Smoke Rose Up Forever) would link to Special:Booksources/1892391201. If you just go to Special:Booksources, you get a form, and can type in an ISBN.

Another useful tool, though one more useful for contributors who have a hard-on for data entry than anything else, is xISBN. It picks related ISBNs, given one edition. For instance, given 0441788386 (Stranger in a Strange Land), it returns a list of other editions, in XML or HTML format.

About encoding... the Jules Verne page says it's in Latin-1 (the meta tag in the head reads "content="text/html; charset=latin1"), but the text itself is in UTF-8. (I'm using Firefox, and with the (automatically selected) Latin-1 encoding, titles show up like Le Château des Carpathes; in UTF-8, they show up as Le Château des Carpathes.

First the easy stuff:

grendelkhan> N Spinrad should be merged into Norman Spinrad. Is this the place for that?

Until editing is fully online, sure. I've already done the merge on the home version of the ISFDB, and it'll be available at the next update.

Ahasuerus> What, precisely, is the ISFDB's scope?

I guess this is as good a time as any to point out that the front page of the ISFDB now states: "The ISFDB is a community effort to catalog works of fiction." Since defining either speculative fiction works or speculative fiction authors can be a thorny process, I don't see any reason to form any strict regulations on inclusion. In fact there are already Westerns, Romances, Mysteries, Techno-Thrillers, and even... mainstream works already in the ISFDB. Stephen King fans want to know about all of King's works - even works which are arguable non-SF, like Rose Madder (which then got nominated for the Britsh Fantasy and Stoker awards). Or Neal Stephenson's Baroque Cycle, which is obviously SF to me, but many others consider Historical Fiction.

So long as the ability to assign tags or keywords to works is available, and users can filter results based on their preferences, I don't see any reason (other than explosive growth) to not open up the ISFDB to fiction in general. I think the fact that the database is still called a Speculative Fiction Database will tend to reign in what gets entered, without an explicit non-inclusion policy.

There are really only two ways a specialty database (or reference book for that matter) can go: Set strict boundaries (and then argue over what side of the line a work falls into), or slowly push the boundary outward over time. I think we've been moving that boundary for some time now.

Ahasuerus> Next, what do we do with foreign language publications?

I think, in general, that there is still an English language bias to the ISFDB. For authors whose first publications are English, I think things are pretty straightforward: the title of the work is in English, and the publications of that work are listed in their translated language, with the annotation (Latvian Edition), or whatever. I don't have any good examples online now, but at the next update check out the J. K. Rowling bibliography for an example of this.

For authors whose publish in a foreign language first (say Stanislaw Lem), the issue is thornier. The correct way would be to list the title in it's original language and add a variant title in the reader's language. The IMDB sort of does this (look up _House of Flying Daggers_), although the variants are always listed in English as far as I can tell.

This has been done in the ISFDB to a small degree (the best example would be the bibliography for Jules Verne (http://www.isfdb.org/cgi-bin/ea.cgi?Jules%20Verne), in particular the Voyages Extraordinaires is done this way, although others of his books have not.

(It should also be noted that TAMU has an older version of MySQL running that doesn't support the LATIN1 character set, so some titles are not displaying properly there yet).

The Strugatsky example illustrates why we wouldn't want to include Cyrilic publications. But the J. K. Rowling example illustrates why we would - if your native tongue is Portuguese, then neither a Cyrilic nor an English bibliography is really going to suit your needs. A Portuguese bibliography would be best, but do we really expect someone to create specific-language bibliographies for every author in every language?

Goodness gracious me, you leave a project alone in the dark for mere 10 years and when you come back, it's got legs!

Oh well, might as well start with something basic. What, precisely, is the ISFDB's scope? IMS, the original notion was to catalog all known works of speculative fiction as well as all (or at least "many") works about speculative fiction. Even that, rather limited, scope raised questions about translations and other, murkier, things.

However,if the scope is to be redefined to include "all known works by 'speculative fiction autors'", including works in other genres, then we need to define "speculative fiction authors" and that can be tricky. Few people will object to finding a complete biblios of, say, H.G. Wells' novels and Simak's westerns in the ISFDB -- as long as they are clearly labeled -- but is a full list of everything ever edited by Asimov that useful? Silverberg's 70+ soft porn novels? Finally, pretty much everybody from Dickens to Twain to James to Keeler to Oppenheim has written something that had a speculative element in it, so where do we stop? Was Shakespeare a speculative fiction author because of _The Tempest_? How about 80% of all ancient Greek and Roman writers?

Next, what do we do with foreign language publications? All good single author bibliography sites (Vance, Farmer, etc) list them, but do we really want to get in that uncertain area? And what about the books published by foreign language authors that haven't been translated into English? Do we want to have a full list of everything published by Barbet and Lem (diacritics and all)? How about Abe (Kanji support??)and Strugatsky (Cyrilic?) Even it were implementable within a reasonable amount of time, would it be really useful considering that there are perfectly fine online bibliographies in their native languages?

Oh well, now that I have stirred the pot, I'll be taking a leave of absence for a few days. Darn Eddorians, they never give up...

N Spinrad should be merged into Norman Spinrad.

Is this the place for that?

Did you ever use the following sql command (or a similar one)?

select
  title,
  author,
  translator,
  copyright,
  ttype,
  count(*)
from
  titles
group by
  title,
  author,
  translator,
  copyright,
  ttype
having
  count(*) > 1;

It returns more than 1000 rows for me - all possible doublicates.

Regards from Germany

Franz-Leo

Patrick Nielsen Hayden 140> Hey, my anthology New Magics isn't a novel.

Hey, it'll be fixed on the next update.

Alex McLintock 138:> So there is a new MySQL database uploaded - but what about the rest of the data? it still seems to be the April 2004 version even though there was a September release.

Answer: Both the tools and the data were updated Jan 9. I've added last modified dates to both pages as well.

need a computer graphics person to make posters/anarchist type

Hey, my anthology New Magics isn't a novel. You've already got it listed in the anthologies; the "novel" listing is superfluous and wrong.

In 1979 Charles Sailor's Avon publishing of The Second Son was noted to be a forthcomig MGM movie. What happened to the screen play? Mr. Sailor and the book seemed to disappear after that. Rumor has it the Catholic Church bought the rights and sat on it.

So there is a new MySQL database uploaded - but what about the rest of the data? it still seems to be the April 2004 version even though there was a September release.

In answer to: Alex McLintock 136
12-23-2004 05:39 AM ET (US)
 
There's one there now, see the link 'MySQL' on the front page.

Is there a MySQL version available for download. It seems to have been talked about but I can only find the old "pre-compiled" version.

I note that the forthcoming books section is far more extensive than I thought. However it is very US centric and thus useless for me as I am mostly interested in UK publications. Is there a way I can help contribute UK publications to the ISFDB forthcoming books?

I am considering doing a system for cataloguing SF books owned by collectors. This may or may not have the ISFDB as the back end data source. Has anyone come across such a system or would they find it useful?

I'm discussing it here: http://www.livejournal.com/community/gopherit/3785.html

As a side issue to this I would be producing a database of review books received as well.

Please note that this email address is no longer in use. It has
changed from ending @onetel.net.uk to end @onetel.com. For example joe_bloggs@onetel.net.uk has changed to joe_bloggs@onetel.com
                                                  Please resend your message to the correct (@onetel.com) address. Your email had as subject line:
    ISFDB

I just found this great page!
I feel I've to say a big thanks for the founder and all of the authors! Nice job!
I would like to ask something about the magazine sizes, I don't know what they mean?! Pulp size, bedsheet size, digest size. What they mean in inches? Could anyone answer me? Thank you!

DNS problems were solved last week. I've got a new email address at alvonruff@gmail.com, and there is now a detailed blog for the ISFDB at isfdb.blogspot.com.

Have some fun play mobile java games and feel the rush!!!

DNS for isfdb.org seems to be wonky right now, but the direct TAMU address still works...

http://isfdb.tamu.edu/

The entire web site at www.isfdb.org appears to have been down for a while now. Is this just a temporary problem?

Looks like you still can't send me email, I can't get my email, and I can't upload any files to the system. Under investigation.

I resubmitted my new book details by email to <avonruff@isfdb.org> but recieved the following
  ----- The following addresses had permanent fatal errors -----
<avonruff@isfdb.org>
    (reason: 554 <avonruff@isfdb.org>: Relay access denied)

   ----- Transcript of session follows -----
... while talking to postman.isc.tamu.edu.:
>>> DATA
<<< 554 <avonruff@isfdb.org>: Relay access denied
554 5.0.0 Service unavailable
<<< 554 Error: no valid recipients
The message in question had the Excel file attached and this could have been the cause. Given the editor's previous comments about email what am I doing wrong?

A correction to my listing: my business is in Slidell, Louisiana. Thanks,

Ruth

Looking good!

I have a suggestion for navigation improvement: if the title of the periodical on pages such as http://www.isfdb.org/cgi-bin/pw.cgi?10448e could link back to the periodical's index page (in this case, http://www.isfdb.org/if.html), then that would be handy.

Edited by author 06-04-2004 11:16 AM
Just a couple of notes:

1. There are still some peculiarities with the new Python scripts that are due to reusing the old ISFDB data files. They could be fixed, but would result in a performance drop. These will mostly disappear after the move to MySQL, so just bear with them for a while.
2. The Excel spreadsheet is really intended for people who are itching to submit large quantities of data. They require only a small amount of work on my part to integrate, so they serve as a fine workaround to data submissions being offline while I work on the tools. Trying to make them more complex by adding support for magazines, anthologies, collections, and data corrections would be just as much work as adding the support to the database, and would just slow done completion of the more preferred solution. So I don't really intend on expanding the scope of the spreadsheet.
3. I haven't had a lot of time lately, so there's lots of email that hasn't been responded to, especially data corrections. I'm on my summer schedule now, so I have about one day a week I can spend on the ISFDB, and I'll be using that to finish off the tools. If you don't hear from me, rest assured that I've always been a poor correspondent, and that it's nothing personal.
   

Hey Al, I've just had a chance to play with the new Python powered version of the site, and some of the new Data you've imported recently, and I just wanted to say Kudos ... things seem to be working really well, but I did notice one little thing I wanted to point out: For some reason some authors are showing up with a blank psuedonym credit in the Name search results. For example a Name a search for "Asimov" lists...
     * [as ] Isaac Asimov
as well as...
     * Isaac Asimov

(likewise for "Robert Silverberg")


Also...


I (and it looks like a few other people) have a couple of questions/comments about submissions...

1) The Excel Spreedsheet looks good, should people just email it directly to you after filling it in?

2) How do you want to handle corrections? should people send you a copy of the XLS with the existing ID and the updated data? What about Author de-duping (ie: "Asimov" is "Isaac Asimov"; "Arthur Charles Clarke" is "Arthur C. Clarke" ; etc...) and Story de-duping (ie: "-That Thou Art Mindful of Him!" is "-- That Thou Art Mindful of Him!" is "That Thou Art Mindful of Him!", etc...) ?

3) The XLS mentions that it can't be used for story titles within publications ... so how would you like people to submit magazines/anthologies? (or for that matter, Author bio info?)

(since it looks like you aren't going to be accepting submissions using the old data file formats anymore, you probably want to gut the "howto.html" section of the FAQ, and replace it with some updated info regarding the new XLS submission mechanism)

Add to listings of Galaxy SF 10/65, "The Age of the Pussyfoot," (part 1 of 3), by Frederik Pohl, and to Galaxy SF 12/65, "The Age of the Pussyfoot," (part 2 of 3).

Peter,
Does this indicate that the ISFDB is open for submissions again?
Derek
My life is not nearly so good as it appears on my website.
http://www.members.optusnet.com.au/~clipstone
< replied-to message removed by QT >

Add to Deborah Wessell:

* The Cool Equations, (ss) Universe 2, ed. Robert Silverberg & Karen Haber, Bantam Spectra 1992

http://www.locusmag.com/index/s783.html
[scroll down]

'Deborah Wheeler' = Deborah J Ross

http://www.sff.net/people/deborahjross/

Al von Ruff wrote (msg 110):
"...a complete rewrite of the ISFDB is underway, and data submissions will be a part of the system. The new engine should be online in a couple of months, and data submissions should be online by the end of summer."

Good news!

You're probably alrady addressing these, but here are a couple of suggestions:

1) have the search box come up as the default first-entry space, on the front page, a la Google.

2) when the author's name typed in the search box exactly matches a name in the db, go directly to that name's page, as opposed to a "selection page" consisting of the name you just typed in.

Minor refinements, but nice to have.

Keep up the good work!

Cheers -- Pete Tillman

I put my first book on the database in 2001 but can't remember how. I want to add my second book can someone tell me how to do this? I filled out the Excel form but what do I do with it? The FAQs and data input rules seem comprehensive but also incomprehensible!

Listing Error, Galaxy Feb 1952,

Hope this is the right place to report this.

Article by Robert A. Heinlein mis-identified as a poem.

Several things:

(1) My piece 'Aliens' of the Past (Amazing, 1991) is shown as short fiction. It's a fact article (one of the science columns I did for Amazing in the early 90s).

(2) I'm in the process of updating (and computerizing) Mike Ashley's comprehensive index to Astounding/Analog (with Mike's enthusiastic permission). I'm going to use the updated database as the basis of an article for Analog's 75th anniversary (Jan 2005). Stan Schmidt has expressed interest in possibly putting the database online eventually at the Analog web site, and it might be another SF resource of interest.

(3) I'm also putting together the articles on "Venus", "Mercury", "Alien Worlds", and "Comets & Asteroids" for Gary Westfahl's forthcoming SF themes encyclopedia. Suggestions as to classic stories or reviews would be most welcome!

You have an impressive site that is a valuable resource for the SF community as a whole.

Steve Gillett
(for reply, please use gillett@alumni.caltech.edu)

Just thought I'd add one to the ISFDB for you:

Marion Zimmer Bradley's Omnibus (though only 2 of the 3 in the series are in this book) titled, "Oath of the Renunciates". From what I can find about it, it was published as a SFBC edition by Nelson Doubleday, Inc. of New York, copyright 1983. I found this descriptor of it's publication on Locusmag website:

SFBC #3651, Jan ’84 [Dec ’83], $7.50, 593pp, hc

Like I said, it contains the first two of the three Renunciates series: "The Shattered Chain" and "Thendara House". Hope this helps, and thanks for the FANTASTIC database site!!

Matt Leavitt
Avid SF Reader

> How do we find out more about this?

Send me an email at avonruff@isfdb.org

> > User's used to be able to add titles to an author's list.
> > Why is this no longer possible?
>
>There were *so* many reasons :)
>
>At any rate, a complete rewrite of the ISFDB is underway,


How do we find out more about this? I am seriously considering re-doing our own version of the ISFDB for my reviews database but if you are refactoring the original ISFDB then we might like to help.

Alex McLintock
Editor DiverseBooks.com
Software DEveloper http://www.OWAL.co.uk/

> User's used to be able to add titles to an author's list.
> Why is this no longer possible?

There were *so* many reasons :)

At any rate, a complete rewrite of the ISFDB is underway, and data submissions will be a part of the system. The new engine should be online in a couple of months, and data submissions should be online by the end of summer.

User's used to be able to add titles to an author's list. Why is this no longer possible? Seems like it would help keep the lists accurate and take some of the load off you.

> Do you intent to update the data archives also?

I'll do another update this weekend with a first round of
fixes, and update the data archive as well.

Do you intent to update the data archives also?

Regards from Germany

Franz-Leo

1] A new update went up yesterday.
2] I'm still working on an all-SQL version of the site
3] I'm still weeding through email, so I haven't caught
   up with the .xls files discussed below.

isfdb.org seems to be working fine but there's been no updates since "Sun May 11 19:37:26 2003". What's the story?
--
Phil

isfdb.org seems to be down now. I hope this is temporary.

Any update on the converstion to SQL tools, since June?

I have the same question. I have tried to submit by sending an XLS form directly to avonruff@ifsdb.org and received no reply.

I have since found a number of minor deficiencies in the ifsdb; if users could easily submit suggested corrections, I suspect that the database would be steadily improved.

This is not meant to be critical. I find the ifsdb in its current state an invaluable resource.

Thanks
Eric

>
< replied-to message removed by QT >

Hello! I was wondering if there was a way to submit the contents for collections and anthologies using the .XLS form and I'm just too dumb to figure out how to do it. :-) If not, how can I do so?

(There are a number of collections in the ISFDB which I own which I would be happy to submit the contents of, but I do not know how -- or even if you're currently letting random strangers submit data at all....)

Thanks!



dhevek at yahoo dot com

A minor error:

Alexis A. Gilliland's novel, The Lord of the Troll-Bats (1992), is the third of the Wizenbeak series, not a stand-alone.

Sorry for the offtopic, but where and how can one report things such as that the the author of "Rats, Bats and Vats" is misspelled as "Eirc Flint"? The form for corrections is gone (though I never noticed any of my submissions getting through :-( and I didn't find anything about it even in the FAQ.

jan dot vanek dot jr at seznam dot cz

Perhaps you might update the list of Aurora winners to include recent years.

You might also wish to list in your database articles published in The Voyageur, an important Canadian fanzine (winner of the Aurora award for best fanzine for the past three years). Its web site is http://voyageur.idic.ca.

Working on programming for Torcon, people come to us and claim all sorts of things about their own importance. I've been counting hits in ISFDB to see how credible such claims are.

QT - Alex McLintock wrote:

>I am wondering whether anyone has converted the ISFDB to SQL and
>whether I have to write my own tools for it.
>
>
Actually, that's what I've been working on during this quiet period with the ISFDB. I've made a few minor
changes to ccpubs, ccauthors, and cctitles to generate output in slightly different format (primarily preserving
note text, instead of using on offset). There is a new set of utilities and sql scripts which will then create an
SQL database from this data. Included are some python CGI scripts that allow you to do more powerful
queries than can currently be performed with the ISFDB.

It's not available in a release yet, since it's still under development, but I could put something together if
you'd like.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Al von Ruff
Editor, Internet Speculative Fiction DataBase: http://www.isfdb.org

Thanks Franz.... where can I find the download of this?

My DBA is a SQL Server fan so MS Access will keep him happy.
(I'm using MySQL or maybe PostgreSQL on the server)

I am wondering whether anyone has converted the ISFDB to SQL and whether I have to write my own tools for it.

The latest version of my standalone Viewer is able to
convert the ISFDB files into a MS Access database.

Regards from Germany

Franz-Leo

Is this the best place to discuss ISFDB? I'd prefer some kind of mailing list.

For instance I have been considering doing a "forthcoming books" for the UK and would prefer to team up with ISFDB to share US and UK info.

Do we chat here?

Alex
http://news.DiverseBooks.com/

Hi folks,

I'm going to be investigating comparing the ISFDB author and title list to our own author and title list to identify where we have titles you don't know about.

I am wondering whether anyone has converted the ISFDB to SQL and whether I have to write my own tools for it.

Alex McLintock
http://news.DiverseBooks.com/

QT - steve reynolds wrote:

>For each issue I have for sale I
>have placed a link to the appropriate page on your database.
>
>Does this cause you any grief or am I doing anything illegal
>such as breaching copyright?
>
>
I have absolutely no problems with deep linking.

Now, about those issues of Analog that are missing from my collection...
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Al von Ruff
Editor, Internet Speculative Fiction DataBase: http://www.isfdb.org

Al
I have recently uploaded my website "Dynamic Pulps" at http://www.reynoldswitham.dsl.pipex.com where I am selling my collection of SF magazines. For each issue I have for sale I have placed a link to the appropriate page on your database.

Does this cause you any grief or am I doing anything illegal such as breaching copyright?
Regards Steve

Relation problems.

I have added a "compiled data" to "Access table" converter
to my viewer. In this module I wasn't able to create the
relations due to invalid or duplicate data. The added test
revealed the following problems.

Duplicate Publication Tag: "BTSFTGLNC1971" ---> "BTSFTGLNC1971_01"
Duplicate Publication Tag: "IMPULSEJUL1966" ---> "IMPULSEJUL1966_01"

The title tags in the following error messages are the offset into the compiled data from 11. May

Missing award tags / Bad award tag in title record

Invalid Relation ("7853", "rewarded for", "007BDEC1") not added to "AwardTitleCombinations"

Missing publication tags / Bad publication tags in title record

Invalid Relation ("BRWNNV2000", "contains", "00A358C6") not added to "PublicationTitleCombinations"

Invalid Relation ("BYNDTHNGW2003", "contains", "000D44C3") not added to "PublicationTitleCombinations"

Invalid Relation ("CRCNP1996", "contains", "002CB0AE") not added to "PublicationTitleCombinations"

Invalid Relation ("DRGNQNTT2003", "contains", "00350807") not added to "PublicationTitleCombinations"

Invalid Relation ("DRGNSGHT1998", "contains", "00350C6A") not added to "PublicationTitleCombinations"

Invalid Relation ("EDODZ2003", "contains", "0038D891") not added to "PublicationTitleCombinations"

Invalid Relation ("EMEYLGRN1998", "contains", "0039AF1E") not added to "PublicationTitleCombinations"
Invalid Relation ("EMEYLGRN1998", "contains", "0039AF7E") not added to "PublicationTitleCombinations"

Invalid Relation ("EVOLUTIONUK2002", "contains", "003BC51E") not added to "PublicationTitleCombinations"

Invalid Relation ("FRMBK82002", "contains", "00445A4F") not added to "PublicationTitleCombinations"

Invalid Relation ("GHSTSYSTRD2002", "contains", "004735AC") not added to "PublicationTitleCombinations"

Invalid Relation ("Greenwood Press", "contains", "009E6DA5") not added to "PublicationTitleCombinations"

Invalid Relation ("HSOFCHANS2002", "contains", "004FFB8F") not added to "PublicationTitleCombinations"

Invalid Relation ("IGNORANT1998", "contains", "005246A5") not added to "PublicationTitleCombinations"
Invalid Relation ("IGNORANT1998", "contains", "00524703") not added to "PublicationTitleCombinations"

Invalid Relation ("ILLIUM2003", "contains", "00525640") not added to "PublicationTitleCombinations"

Invalid Relation ("LNBYNGHTBF1962", "contains", "000B4501") not added to "PublicationTitleCombinations"
Invalid Relation ("LNBYNGHTBF1962", "contains", "003A9108") not added to "PublicationTitleCombinations"
Invalid Relation ("LNBYNGHTBF1962", "contains", "0042006F") not added to "PublicationTitleCombinations"
Invalid Relation ("LNBYNGHTBF1962", "contains", "00456131") not added to "PublicationTitleCombinations"
Invalid Relation ("LNBYNGHTBF1962", "contains", "00482B31") not added to "PublicationTitleCombinations"
Invalid Relation ("LNBYNGHTBF1962", "contains", "0053D555") not added to "PublicationTitleCombinations"
Invalid Relation ("LNBYNGHTBF1962", "contains", "00596516") not added to "PublicationTitleCombinations"
Invalid Relation ("LNBYNGHTBF1962", "contains", "00610ACD") not added to "PublicationTitleCombinations"
Invalid Relation ("LNBYNGHTBF1962", "contains", "00711C7E") not added to "PublicationTitleCombinations"
Invalid Relation ("LNBYNGHTBF1962", "contains", "0095736D") not added to "PublicationTitleCombinations"
Invalid Relation ("LNBYNGHTBF1962", "contains", "00975D0B") not added to "PublicationTitleCombinations"

Invalid Relation ("LTMTSV199X", "contains", "00A232B2") not added to "PublicationTitleCombinations"

Invalid Relation ("MMTHBKSC2002", "contains", "0065783E") not added to "PublicationTitleCombinations"

Invalid Relation ("MORGAWR2002", "contains", "006C9BE7") not added to "PublicationTitleCombinations"
Invalid Relation ("MORGAWR2002C", "contains", "006C9BE7") not added to "PublicationTitleCombinations"

Invalid Relation ("NGNCITY2003", "contains", "003A7BFC") not added to "PublicationTitleCombinations"

Invalid Relation ("NKDMPR2003", "contains", "006EAEA4") not added to "PublicationTitleCombinations"

Invalid Relation ("NWMGCS2003", "contains", "006FE4F9") not added to "PublicationTitleCombinations"

Invalid Relation ("PRTCALL22000", "contains", "00641745") not added to "PublicationTitleCombinations"

Invalid Relation ("SHIPOFLS1999", "contains", "008C641C") not added to "PublicationTitleCombinations"

Invalid Relation ("SLVATION19XX", "contains", "008675E0") not added to "PublicationTitleCombinations"

Invalid Relation ("SMTHNGFTNT2003", "contains", "009021BE") not added to "PublicationTitleCombinations"

Invalid Relation ("STRNGBNST2002", "contains", "00956CBB") not added to "PublicationTitleCombinations"

Invalid Relation ("THFTHSRCRS2002", "contains", "003F3E1A") not added to "PublicationTitleCombinations"

Invalid Relation ("THGTHRNGUK2003", "contains", "00464AE5") not added to "PublicationTitleCombinations"

Invalid Relation ("THISSTROM2000", "contains", "0016E8C5") not added to "PublicationTitleCombinations"
Invalid Relation ("THISSTROM2000", "contains", "009B5D97") not added to "PublicationTitleCombinations"

Invalid Relation ("THSRPRSCN2002B", "contains", "00A3ACB3") not added to "PublicationTitleCombinations"

Invalid Relation ("THSTNDNGD2002", "contains", "0092F929") not added to "PublicationTitleCombinations"
Invalid Relation ("THSTNDNGD2003", "contains", "0092F929") not added to "PublicationTitleCombinations"

Invalid Relation ("THXROADS2002B", "contains", "002C9928") not added to "PublicationTitleCombinations"

Invalid Relation ("THYFTRNTY2002", "contains", "003CB44C") not added to "PublicationTitleCombinations"

Invalid Relation ("TMWSTG2003", "contains", "0065EFCC") not added to "PublicationTitleCombinations"

Invalid Relation ("TSHDWSLGT2002", "contains", "008BA63D") not added to "PublicationTitleCombinations"

Invalid Relation ("TTRTHSA1998", "contains", "00A102F2") not added to "PublicationTitleCombinations"

Invalid Relation ("VARTHMNWT19XX", "contains", "003B2957") not added to "PublicationTitleCombinations"
Invalid Relation ("VARTHMNWT19XX", "contains", "00A448D7") not added to "PublicationTitleCombinations"
Invalid Relation ("VARTHMNWT19XX", "contains", "00A44967") not added to "PublicationTitleCombinations"

Invalid Relation ("VNNGSMPR2001", "contains", "003B790A") not added to "PublicationTitleCombinations"

Invalid Relation ("VYGSHDWMN2002B", "contains", "00A5EDFD") not added to "PublicationTitleCombinations"

Regards from Germany

Franz-Leo

QT - Franz-Leo Chomse wrote:

>Bad link on ODP data
>
>The viewer entry points to the original HTML data not to
>the special viewer version
>
>
Sorry. Fixed.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Al von Ruff
Editor, Internet Speculative Fiction DataBase: http://www.isfdb.org

Bad link on ODP data

The viewer entry points to the original HTML data not to
the special viewer version

Regards from Germany

Franz-Leo

>If you have a need to start going down the real database path
>now, you should feel free to blaze a new trail - we should just
>keep in sync as things progress.

The main reason for the unique tag are updates. As long as you
recreate the database each time you get new data, the identifier
fields provided by your database is all you need.

At the moment when you allow more than one person to update the
records or when you need to distinguish between new or updated
records, you need "stable" identifiers. At the moment I use a
combination of title and CRC check sum to identify unaltered
records.

To be able to distinguish between updates and inserts I need to
know whether the order of titles in the ascii files can change,
i.e. whether new records are always added to the end of the file. (If they are created from a database table with an autoincrement key, this would mean that they are exported ordered by this key).

I can add a conversion module to the viewer which creates an
ACCESS database from the dbase.compiled files, if you mean that
there is need for it. Due to licence reasons and OS language
dependencies the jet engine has to be installed already.
(Free download from the Microsoft side)

Regards from Germany

Franz-Leo.

QT - Hoss wrote:

>When i first found the isfdb, I was tempted to write a script to
>make it easier to submit magazine data ... but I held off, first
>because of your online submission form, and then when you took
>that offline, i started thinking about it again .. but i held
>off because it loked like you were planning to change the format
>to XML.
>
>
Here's where I'm at on XML these days: When we implemented some new tools a couple of years
ago at work, we implemented them in XML for all of the usual reasons. One is a task-tracking system
which keeps track of the various tasks engineers are working on, allowing tasks to be prioritized by
management, and for engineers to update effort estimation. Tools then calculate when a specific task
will be completed (as well as other things). Each task is represented by a single XML file. In general
this tool works well, and is fast, but certain administrative tasks are difficult because everything is a
*file*. Since it been running a couple of years, there are *lots* of files, and lots of permissions to track, and
it's hard to set up new users, and it's hard to set it up elsewhere for another project, because all these
files have to be setup just so.

The second tool is used to track the usage of our... product. We build an internal used product, so to track its
usefulness, we get a record containing information everytime the tool is used by someone. The
data is sent to us in XML format, and then stored in a single monthly file. In general, this is reliable,
and XML allows us to add new fields without having to rewrite parsers and so on. The down side is
that there are hundreds of megabytes of XML data now, and doing simple searches takes double-digit
minutes to perform. This is now migrating to an SQL database.

So, the upshot is: I think XML is great for formating data when it's being transmitted. Having access to
Amazon.com records in XML format has provided a great way to rapidly fill in books in the ISFDB,
and I think that XML is a great format for submitting new data. I don't however, think that XML is
a good way to store lots of data, because you either wind up with a small number of really large files
(which is what we have now with the ISFDB, just different format for XML tags), or you wind up
with a huge number of small files (which is difficult to manage for other reasons). So what I'd like to
have is an SQL server, where data can be submitted in any number of formats, but the preferable form
for small submissions would be XML.

So I think that a script that allows people to enter magazines, and then fires off a piece of XML
email (or follows some other transport to) someplace, and having a backend parse it and push it into
the database is a perfectly reasonable thing to do. This is probably pretty easy for new zines, and
if you wanted to work on something along those lines, I'd work on the backend for it.

It'll get more difficult when it comes to corrections, but we can cross that bridge later.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Al von Ruff
Editor, Internet Speculative Fiction DataBase: http://www.isfdb.org

QT - Franz-Leo Chomse wrote:

>It doesn't matter how the tag is build, for a database you
>just need an unique identifier (per table or globally) if you
>want to maintain references to the data.
>
>
In order to reference a specific title, the SQL layout has a 'pubid' which is a unique AUTO_INCREMENT
type. The titles were unmerged before insertion into the database, so each title in each publication is unique
and has its own unique ID. However, when the data is exported into ISFDB format, the pubid is dropped
- at least for now.

So to reference the contents of a particular publication, a SELECT is done to find the set of titles with the
ISFDB publication tag. If modified, titles are updated via the pubid field.
>P.S. In database theory you have the rule that a record
>identifier should not be based on the contents of the
>record, since the identfier has to survive any change to
>the data.
>
>
Once created, ISFDB tags are immutable even if the basis for the
the tag changes. On the other hand, every record in the SQL version does have an ID that's created by the database and has no relationship to the data as you say. We'll probably keep the tags as well, because it reduces error when referencing a record from an HTML page.

Let it be said that when it comes to an SQL version of the database, nothing is set in stone. My plan was to create a simple database
layout that mostly replicated the ISFDB layout, just to help in the daily maintenance - and then to start modifying it to become a real database.

If you have a need to start going down the real database path now, you should feel free to blaze a new trail - we should just keep in sync as things progress.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Al von Ruff
Editor, Internet Speculative Fiction DataBase: http://www.isfdb.org

>As I understand it, with the exception of data that has been
>mass imported from other DBs, the IDs have allways been manually
>created -- which is usefull so that they can follow a logical
>pattern (ie: all issues of the same magazine tend to have the
>same prefix, books and magazines tend to have the publication
>year in the id, etc...). I would assume (and encourage) Al to
>continue that practice even if he created a new DB format, or a
>GUI for entering data.

The problem is that at the moment only the awards and the
publications have a tag which is supposed to be unique.

Authors have a canonical name which needs to be made unique
by adding an extension.

Titles are identified by their offset into the compiled
data file "tables.dbase". In the source files the key is
the title but this is NOT unique. Thus introducing a title
tag into the new structure is similar to a mass import.
It doesn't matter how the tag is build, for a database you
just need an unique identifier (per table or globally) if you
want to maintain references to the data.

P.S. In database theory you have the rule that a record identifier should not be based on the contents of the record, since the identfier has to survive any change to the data.

Regards from Germany

Franz-Leo

Franz...

> But the question of a unique identifier for the titles also
> occurs in the data entry task as soon as you use a central
> window for entering the contents of a magazine or when you
> want to adjust to external update information.

As I understand it, with the exception of data that has been mass imported from other DBs, the IDs have allways been manually created -- which is usefull so that they can follow a logical pattern (ie: all issues of the same magazine tend to have the same prefix, books and magazines tend to have the publication year in the id, etc...). I would assume (and encourage) Al to continue that practice even if he created a new DB format, or a GUI for entering data.


Al...

When i first found the isfdb, I was tempted to write a script to make it easier to submit magazine data ... but I held off, first because of your online submission form, and then when you took that offline, i started thinking about it again .. but i held off because it loked like you were planning to change the format to XML.

The main idea I had was to allow users to type up just the "TOC" (ie: list of story titles) and then let the script apply the regular merge rules, looking for stories/essays/etc... with similar (or the exact same) titles, and ask "Did you mean: ___?", and if the answer is no, prompt them to enter the items "type", and specific data -- applying the same "Did you mean: ___?" logic for Authors, Artists, Reviewers, INterviewers, etc.

I think this still might be useful, and probably even easier in a Relational DB backed data-entry system. You might consider including something like this in your design.

>To use a DB for the web site will require more changes,
>including normalizing the database. Doing that is further off.

For me it is the next step after the optimization of the HTML
generator in the Viewer as part of the optimization of the load
process.

But the question of a unique identifier for the titles also
occurs in the data entry task as soon as you use a central
window for entering the contents of a magazine or when you want
to adjust to external update information.

Regards from Germany

Franz-Leo

QT - Franz-Leo Chomse wrote:

>Regarding the switch to MySQL
>
>The major problem by switching to a DB are the missing
>tag values for the titles.
>
There are two different use cases for using a DB: data entry and web site. The current work addresses using a DB for data entry. Editing the data files in a text editor is clumsy and time-consuming due to the size of the files, and makes it impossible to have mulitple people work on the database simultaneously. Moving to a DB for data entry is pretty easy, since it just means replicating the same tables in use with the compiled version of the data files (minus the indices). Just requires writing some application code to do the actual editing and administration.
To use a DB for the web site will require more changes, including normalizing the database. Doing that is further off.
 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Al von Ruff
Editor, Internet Speculative Fiction DataBase: http://www.isfdb.org

Regarding the switch to MySQL

The major problem by switching to a DB are the missing
tag values for the titles. The easiest ystem for tags
would be a 32-bit integer value. By giving each source table
a unique start values with gaps between the different
start values big enough to avoid duplicates in different
tables the key values are still unique when the tables
are merged.

Anthologies 1.000.000 ... 1.999.999
Artwork 2.000.000 ... 2.999.999
Books 3.000.000 ... 3.999.999

...

Regards from Germany

Franz-Leo

Warning for Windows Users.

if you try to extract the compiled data files with
a Windows archive tool like WinZip assure that flags
like "smart CR/LF conversions" are disabled.

I have been debugging my viewer for two days to find
out. why the program did no longer work after the last
refactoring step. But I just found out that the lines in the data files were terminated by a CR-LF combination and not a LF combination. This change made the offsets in the reference files invalid which in turn returned garbage in the title list which triggered an endless loop in the HTML generators.

Regards from Germany

Franz-Leo

>This used to be handled with symbolic links, but that's not
>going to be portable in the Windows work you're doing. We also
>don't want to maintain two files. I don't particularly care
>which file is the master file, and which file is the copy. If
>the current setup isn't working for you, I'm open to
>suggestions.

It doesn't matter as long as it is not changed again.

Regards from Germany

Franz-Leo

> I know, but the queries still generate references to sfdbase.html and not index.html.

Yes, hence my remark "Some day we'll obsolete the need for sfdbase.doc"

In essence, sfdbase.doc and index.doc are the same file. They are both needed: sfdbase is referenced by the tools, index is referenced whenever someone goes to www.isfdb.org/ (or whatever the root URL is on your system).

This used to be handled with symbolic links, but that's not going to be portable in the Windows work you're doing. We also don't want to maintain two files. I don't particularly care which file is the master file, and which file is the copy. If the current setup isn't working for you, I'm open to suggestions.

>sfdbase.doc: This is a copy of index.doc. You need the new
>version of dbase.mk in frame.tar.gz to make the copy during the
>installation. Some day we'll obsolete the need for sfdbase.doc

I know, but the queries still generate references to sfdbase.html and not index.html.

>printf(" [Main Menu]", HTMLLOC);

The old script copied sfdbase.html to index.html. There is still
a reference to it in htmls.mk.

>htmls/sfdbase.html
Thus the change has side effects for my viewer program.

Regards from Germany

Franz-Leo

isfdb.jpg: I put a new version of html.tar.gz up. It contains the missing image.

sfdbase.doc: This is a copy of index.doc. You need the new version of dbase.mk in frame.tar.gz to make the copy during the installation. Some day we'll obsolete the need for sfdbase.doc

Jeff: I sent email off to the Java JODBC originator but haven't heard back. I've been working on a MySQL version of the database (publications only so far), and have been modifying the database compilers to also output versions that can be imported into MySQL. At present I've got a C API for making queries, but will eventually migrate to Python once everything is working. Once I can start editing the database via a Web interface (instead of using a text editor), I'll package it up.

sfdbase.doc is missing in the data files
isfdb.jpg is missing in the data files

Regards from Germany

Franz-Leo

Could the person who wrote the Java ODP->JODBC translator give me a shout at jeffb@isc.tamu.edu, or post an update on the forum?

Thanks,
Jeff

I'll have more on the data submission plan later, but for now will just say that I'd prefer to devote the majority of my time on building a robust data submission system. If you have bulk data to integrate, I'll take that (and did so this weekend), but I don't want to get bogged down in minor tweaks and fixes yet.

: How/where should people submit corrections to incorrect data
: in ISFDB? I'm not talking about bulk data, just obvious
: errors that people notice when they browse ISFDB.

Acctually, that's a really good question ... now that Al's email isn't on the site anymore, it's not clear how any data submissions should be made.

How/where should people submit corrections to incorrect data in ISFDB? I'm not talking about bulk data, just obvious errors that people notice when they browse ISFDB.

The ISFDB used to point here, but there's no longer a link on the current page. It'd be nice if the FAQ mentioned something about where people should report bugs/corrections.

Thanks,
Lloyd Lee-Lim

Al:

Thanx for the update ... work just got crazy, but hopefully sometime this weekend I can do a fresh install of the latest code/data and start submitting some updates from my latest Mag buying frenzy.


Everybody:

Now that this forum exists, I figure I might as well mention that over the past few months, I've been writting a bunch of command line perl scripts that I've found usefull for dealing with the compiled database files. I don't know how useful other people might find them, but I posted one online a while back...
  http://perlmonks.org/index.pl?node_id=203217
...the rest aren't really documented very well just yet, but if people find stuff like this useful, I can bronze plate them and post them as well.

Hoss:
 
1] Data should now be online. Note that the htmls directory
   has been drastically changed to support uploads to TAMU,
   so it's possible that new problems may be encountered.

2] I've addressed your portability issues, except for the
   mkdir -p one.

Franz-Leo: I was pulling your leg.

On 9 Apr 2003 01:11:01 -0000, QT - Al von Ruff <qtopic+18-cA5ncMsbYm3Xj@quicktopic.com> wrote:
>--QT-------------------------------------------------------------
> Reply by email or visit
> http://www.quicktopic.com/18/H/cA5ncMsbYm3Xj/m64
>-----------------------------------------------------------------
>
>> Your html_print1_author deals with a collaboration in
>
>> au_actual but not with a colaboration is au_pseudo
>
>
>
>Holy crap! After eight years, someone is finally doing a code
>review. And - you actually supplied test cases.

The reason most likely is that these cases only occur when you use search.search_file i.e. scan "table.dbase" and not search_file2 which uses "tables.xba". The authornames in the reference file do not have collaborations in the pseudonym part. On the other side the reference contains names ending with an "^". Both facts together raises suspicions about the compilation.

>Invalid composed person name: "Mitchell Burnside Clapp^"
>Invalid composed person name: "Mildred Butler^"
>Invalid composed person name: "Michael Swanwick^"
>Invalid composed person name: "Michael Sumbera^"
>Invalid composed person name: "Michael Moorcock^"
>Invalid composed person name: "Mark Reinsberg^"
>Invalid composed person name: "Lynn Abbey^"
>Invalid composed person name: "Lucy Kemnitzer^"
>Invalid composed person name: "Lin Carter^"
>Invalid composed person name: "Leroy Kettle^"

The trailing '^' produces an empty au_pseudo anyhow, but triggers a range check error in my port.
N.B

in search_file you have to decompose set->se_author and not tmpauthor. Seems like this branch is never used.

And I'm not doing a code review - I'm doing a port. And therefore I need a lot of diagnostic output to get the big picture anyhow.

Regards from Germany

Franz-Leo

Hey Al,

I noticed that the odptools.html page is online now, but it only contains links to the frame, src, & tools .tar.gz files ... there are still no links to data.tar.gz or html.tar.gz

I also noticed that there are still a few portability bugs in the code (I emailed you about them a while back, but I'm sure you had bigger things to worry about trying to find a host/sponsor) ...

1) frame.tar.gz creates a "dbase.acii" directory .. evidently a mistake.

2) I had to change the CFLAGS in the Makefile not to
   include -DCYGWIN. You should probably add CFLAGS to
   the list of Makefile variables people will want to
   customize, and give a list of platforms you use in
   #ifdef so they know what their options are
   (I think i saw CYGWIN, SUNOS, and AIX ... but there
   might be more.

3) tools/info.c assumes sort is "/bin/sort", which isn't
   true on all systems (on mine it was in /usr/bin/sort)

4) dbase.mk assumes bash is "/usr/bin/bash"

5) You might want to use "mkdir -p" prior to copying
   files into the directories specified in the Makefile
   ... It's not that important since the directories
   probably already exist, but it's a little thing, and
   in my case i was setting up a new installation from
  scratch.

> Your html_print1_author deals with a collaboration in
> au_actual but not with a colaboration is au_pseudo

Holy crap! After eight years, someone is finally doing a code review. And - you actually supplied test cases.

I hope you don't want to see the design doc.

Al

Your html_print1_author deals with a collaboration in
au_actual but not with a colaboration is au_pseudo

  au_actual: |Will Murray|
  au_pseudo: |Warren B. Murphy+Richard Ben Sapir|

   au_actual: |Fletcher Pratt|
   au_pseudo: |Irvin Lester+Fletcher Pratt|

   au_actual: |Wilfred Owen Morley+Jacques DeForest Erman|
   au_pseudo: |Forrest J. Ackerman+Robert A. W. Lowndes|

   au_actual: |Forrest J. Ackerman+Robert A. W. Lowndes|
   au_pseudo: |Wilfred Owen Morley+Jacques DeForest Erman|

   au_actual: |Randall Garrett|
   au_pseudo: |Sam+Janet Argo|

   au_actual: |Howard Browne|
   au_pseudo: |Jack Lait+Lee Mortimer|

   au_actual: |unknown|
   au_pseudo: |P. Purzillo+U.R.A. Furball+C. Kitirun|

   au_actual: |Will Murray+Ric Meyers+James Mullaney|
   au_pseudo: |Warren B. Murphy+Richard Ben Sapir|

Regards from Germany

Franz-Leo

There are two possible ways to distribute it.
1. Only the ExeFile
2. Exefile plus Data.
For the second version Al would be the responsible man,
to assure that the online and the offline versions remain
in sync. As already said, at the moment the viewer leaks
memory like hell, but if anyone wants to test it just
give me an E-Mail address which accepts a 1 MB attachment
for the exe-file only or a 12 MB attachment for exe-file plus data(as 2002-12)

It should just be an offline version of the ISFDB. Which
means that it uses the same compiled datafiles to have only
one set of compilation tools. All queries are combined with
a Html Viewer in a single Exe-File, which has to be copied
into the base directory (the one with the dbase.compiled
sub directory). Only the HTML files have to be changed, since I use the dummy protocol "query://" to activate an
internal query and "page://" to activate a local html page.
As a result the fixed HTML pages need to be adjusted, but
this is done in the same way as now: replacing the template
reference with a new one. The basic port from the C sources
to Delphi is done. Now I'm fighting with the memory leaks,
since the original C source does not free the allocated structures (this is not needed when the application is
terminated after each query). Since the viewer should only read from disk, it will be possible to put the base directory with the executable and the two sub directories dbase.compiled(data) and dbase.html(local html) onto a CD.

1] www.isfdb.org now points to the TAMU site. It took a couple of days (and some help from Jeff) for the new nameservers to take effect.

2] The ODP stuff isn't online yet since it requires a bit of restructuring to fit on the new web site. It should show up in a bit.

3] Franz-Leo: What would you like to do with the standalone application? I'm sure that people would like to test drive it.

the proof of concept version of a standalone application
to display the ISFDB compiled datafiles is running. It
is written in Delphi. (As a result you could provide a
CD ROM version of ISFDB). During the conversion I found
that the author index (titles.xba) contains author names
ending with an "^". Additionally there are still entries
with an "," between collaborating authors.

The application is written in Delphi and uses two new
"protocols" to indicate the local HTML pages and queries
on the local copy on the compiled data files. Thus the
only change to the compilation process is a new make_local_htmls script with adjusted replace strings.

Regards from Germany

Franz-Leo

Greetings Al, I just found this discussion board and wanted to submit a few comments/questions...

1) congrats on the new site sponsor

2) I know the temporary url (http://isfdb.tamu.edu/) is up an running, but just in case you it's not apparent from your local network, from where I am http://www.isfdb.org/ no longer even seems to work enough to tell people about the temporary URL... in fact, most of the whois servers I tried don't even acknowledge that the domain is registered.

3) One thing that seems to be missing from the new site are the "tools" and "data" pages to download the various .tar.gz files for building local copies of the DB (they used to be available at http://www.isfdb.org/odp/data/ and http://www.isfdb.org/odp/tools ... but the corrisponding directories don't seem to exist on the new site).

4) You should promote this forum more ... I didn't see any mention of it on the front page, or in either the the "What's New" or "FAQ" sections of your site ... it was only by fluke that I noticed the link to your name wasn't an email address like it used to be.


Again, congrats ... i hope everything works out well with tamu.edu.

hossman/at/fucit/dot/org

No problem: considering the number of authors listed in the bibliography who do not have bios attached to them, why not just set mone to "null", as it were.

Sorry to keep harping on the issue, but I would feel much better about having my biography removed. I've already done what I can to remove from memory the fact that I ever wrote in this genre, and this would help immeasurably.

Hi Al,

Great news that you've found a new home for the ISFDB.

When you're ready to start updating again, please let me have your email address (I tried the SFSite one a couple of weeks back) so I can send the missing 02/03 NYRSF & Interzone details. I could also let you have Asimov's & F&SF if you don't already have the info.

QT - Paul Riddell wrote:

>Just out of curiosity, is there any way an author can have
>his/her Author listing removed from the Database? I know that
>it's supposed to be a comprehensive source, but I know I'd make
>a lot of people in the genre very happy if any reference to me
>were expunged. Myself at the top of the list.
>
>
It's easy to get fooled into thinking that the ISFDB is a collection of bibliographies - it's not. It's really
a collection of publication information. To remove your bibliography would require removing your essay
from the publication information for the Spring 1996 issues of Science Fiction Eye, for example. That
action would make the data for that issue incomplete. The bibliographies are a side effect of the database,
not the focus.

I suppose it could be possible to inhibit the display of the
bibliographies of certain authors, but the
utility seems dubious

Just out of curiosity, is there any way an author can have his/her Author listing removed from the Database? I know that it's supposed to be a comprehensive source, but I know I'd make a lot of people in the genre very happy if any reference to me were expunged. Myself at the top of the list.

I saw on an AOL SF board that ISFDB was back online, which I was glad to hear, and I wondered if you were taking new listings? (I mean updates to my listing.)

During the time the ISFDB has been down, I've been working
on a program to load the ISFDB data files into a local
database. It's a java program, and uses JDBC, so it works
with any platform that supports Java, and any relational
database you can get a JDBC driver for. So far I've tested
it on the following successfully:


Windows XP + Access 2002
Solaris + Sybase 11.5
Red Had Linux + mySQL


I wrote the thing for my own use, but I thought there might
be some interest out there for such a tool.
Anyway, I'm willing to donate the thing, and provide support
for it if anybody is interested.
Not sure how to make it available, though.


If anybody is interested in using or working on it
(would be willing to put it on SourceForge if there
are some interested people willing to collaborate)
let me know. It works now, but needs some refinements for general public usage.


Jim Williams
jamespwilliams@attbi.com

- There is an old mystery from SFPA about Eugene Sue's THE WANDERING JEW. Anther member of the apa sent me an old English translation that has Dagobert saying "Chain up your curb!", but I have never been able to find a French text with anything in the same spot but "Laissez moi en paix!". I have been told that Sue was sanitized in the late 1800s and all the interesting jargon removed - do you have an old edition of this book like the 1845 you list?

- I have myself (as Purple Mouth Press) published two Arthur Machen books that aren't in your database - where would the data be put in?

I think there's a type here:

http://isfdb.tamu.edu/cgi-bin/ea.cgi?Pier_Xanthony

That should be Piers Anthony. :)

QT - Jlewis70452aol.com - Jim wrote:

>Is there a new site up? Also, I would be glad to contribute to
>its upkeep.
>
>
You can temporarily access it at http://isfdb.tamu.edu

The www.isfdb.org domain name should be pointed there soon.

Is there a new site up? Also, I would be glad to contribute to its upkeep.

Jim

Is there a mirror site right now?

"The ISFDB is sponsored in part by Amazon.com. _Generic Adventures_ is written by S.F. Author. To order _Generic Adventures_, click here."


No?

If the UCSD site doesn't come through I maintain a server at Virginia Tech on which I could also host it.

Though I don't know how to get in touch with you outside this forum... You can search for my name on the Virginia Tech directory though if you want to get in touch.

Edited by author 02-24-2003 12:28 PM
Y'know, I've always wondered how awesome resources such as the ISFDB run with so many hits and still manage to be free to the public. Glad to hear there are people out there willing to provide free hosting to keep such information sites alive. Glad to hear it worked out. Now if I could just figure out how you get so much damn info on a free site without hiring a full-time staff. :)

QT - Joe Morris wrote:

>I've been running everything on a small solaris box (no outside
>access right now) and everything works fine, except submitting
>new data. I don't see how the "path" variable ever gets set in
>submit.c (lines 1215 & 1553)
>
>
The "path" variable is a red herring. The real secret to getting
submissions to work is to touch the "NEWDATA" file in the cgi-bin directory. The fopen() calls will fail if it doesn't exist. Hasn't really been a priorty to fix as the data submissions have been
turned off for a while.

The ISFDB is now up and running at Texas A&M University, and will be publically available as soon as the final paperwork clears
through the university.

Is this located in California? If so, California Community colo might be the perfect place for you. http://communitycolo.net/ .. It's for nonprofit organizations. Donations are good but not required. There stuff is at Hurricane Electric. You just drop your box(es) in and they give you an IP address. You can go in and manage your box at any time.

I've been running everything on a small solaris box (no outside access right now) and everything works fine, except submitting new data. I don't see how the "path" variable ever gets set in submit.c (lines 1215 & 1553)

Anybody know what I should do to get it working?
Thanks joe@jolomo.net

Al,

UC San Diego's SF group is willing to host the ISFDB on a dedicated server. I'll drop you an email with the details.

-- Adam Tilghman, UCSD

micropayments may not yet be an option but i would be more than willing to donate $10 to a pay-pal account. Once the account reaches a comfortable cash level to keep the site up for a year, co-locate and remain ad-free. yeah for all :)

sad to here about the isfdb. I really loved it and it is mostly superior to locus. for my most used author's I had downloaded their index pages. If you decide to "puplish" a cd-rom mirror (or copy) I'd love to buy a copy. I do hope things can be arranged. i guess something like geocities would not work either, even though their advertising is slightly annoying. liet me know.
Thanks,
Jim

Bowling Green University's Popular Culture Library is working on creating a online index of pulp magazines. This sounds like there could be some synergy between this site and BGSU. Contact them.

Rick

Are there any traffic statistics that would give us a hint as to the necessary bandwidth required to colo a server?

There are likely several groups on the UCSD campus that might be able to host a server, depending on the load we would see.

tep@sdsc.edu

The local database seems the only way to go in the long turn. But there sould a request be added to the design.
The local database has to be build up incrementally. This
avoids a long download every time, an update is requested.
But it needs either a databse on the backend also, or a
common place where you can download files containing only
the changed records since the last installment.

Regards from Germany

Franz-Leo

Hi

My name is David, and I carry a "similar", though obviously smaller, DB of scifi, fantasy and horror books published in Spanish, cyberdark.net.

And we have the same problem. Currently we have 3500 vistors a day, which means 25 Gb transfered each month, and each six months statistics go double. We'll have to rent a server in the next three months (350 euro / month), because bandwith and SQL queries.

So we're looking for money. I've read in your website donations is not the way, but I thing that for a website like yours it's needed users to help. There's nothing bad in asking money from people to keep your great site running. We're doing that way at this moment, while looking for a final solution. Even more if yout site is GNU.

Go ahead! Your site is a mirror where to see how to do things in the right way for many people.

Regards (and sorry for my English).

I was really sad when I learned that ISFDB was about to close, partially for selfish reasons: I had been using it to find information to complete my own database of SF&F works published in Portugal (currently also on hold waiting for a major change: it's transformation from HTML static pages into a PHP-powered dinamic site), since it's a common thing that our translated collections and anthologies do not come with the original titles of short fiction. And there are other nits to be picked as well, and ISFDB was invaluable in picking them...

I'm more satisfied now that I see that the problem is on it's way to be solved.

In the meantime, would you happen to have your data available as excel tables, or something of the sort? Finding stuff in huge ASCII files is a big pain and I'm dumb as a door when it comes to UNIX-based systems.

You can find my e-mail prominently displayed in my site at http://ficcao.online.pt/enciclopedia

Keep up the good work!...

Al, I certainly wish you good luck with getting this database up and running. Since I discovered your site a year ago, I have been able to update my database of my collection of speculative fiction with all sorts of information I did not have previously for items that have been purged from my collection over the years. Thanks for the great site. I look forward to you being back up and running soon.

Al,
   I hope you are making progress with a new host. If you are still "looking for a home," there may be options connected with our Science Fiction and Fantasy Research Collection here at Texas A&M. If that option interests you, we can discuss directly.

Hal Hall
Curator, Science Fiction and Fantasy Research Collection
Cushing Library
Texas A&M University
College Station TX 77802
Hal-Hall@tamu.edu
979-862-1840

Folks,

I help run what may be the world's only formal non-profit colocation project. We provide free, donation-based colocation for individuals and non-profits. A donation of $50/mo covers our cost for a 1U or 2U rackmount server using an average amount of traffic. You can find out more about us at http://CommunityColo.net/ - and we're now #1 on a Google search for "colocation"! :) We've got lots of happy folks with us, including the Special Olympics. Let me know if we can be of assistance.

Just an update on status - I've gotten a lot of good suggestions for ISPs, but I've also gotten some free hosting offers from a couple of major Universities. We should hopefully see some resolution to things this week.

Al,

If the leads for free or cheap hosting below don't bear fruit, we might work out a solution together. Our SciFan database and Isfdb are good complements: we cover series, web links and forthcoming books better and we have huge thematic coverage, but we don't cover other material such as short stories, articles or specific book editions at all, and we haven't put awards into our database yet.

I'd be glad to run your search queries from our site (and of course cover the bill for them) and output results as embedded links within our own search results, i.e. your search form would point to our search engine, which would output results pointing back to ISFDB and/or SciFan (depending on whether you and/or us have results to show). A side space would feature your logo and explain why we're doing this, and what users can expect from each site.

This would work for us because it would be an opportunity to reach more users (people coming from your site would discover ours and see our results along with yours.) SciFan is still a hobbyist resource at this point but we have an unashamed for-profit perspective (we earn money on affiliate book sales and advertising.) I think this could work for you too since you'd keep the ability to respond to search queries, without having to support the load. And you'd keep your own domain and hosting to serve static HTML pages. Since you don't update the online ISFDB frequently, it makes sense to "bake" your content, not "fry" it for every page request (we fry our pages directly off the database because we update our data on an ongoing basis.)

We run on a shared IIS/SQL Server environment hosted by Innerhost (now owned by Interland.) They're a large and reliable host, and though they're not without outages, these have been rare and quickly fixed in the past 4 years we've been running with them. As of lately, how many search queries have you served per day? What about peak usage?

Let me know whether what I'm proposing makes sense. I could do a page mockup to explain better what's on my mind, and we would need to look at the technical aspect of things as well. My search engine is an ASP-driven HTML form (that I can feed through either GET or POST), I'd need to simply output links to your own pages depending on the user's search criteria. This should be fairly easily automated if your URLs follow structural rules or use Ids.

Whatever route you decide to take, ISFDB needs to stay online. It's been a strong inspiration for us and a useful research tool, and it should remain available to the public.

Best,

Olivier Travers
otravers (at) scifan.com
http://www.scifan.com

FYI, I *needed* to get this set up on my desktop Unix box because I just couldn't survive without ISFDB. The only changes I needed to make from the tarballs were the following:

Change the documented variables in the top-level Makefile then in tools/ccdbase.c commented out the doerror function calls on lines 63 & 73. Everything then compiled perfectly and is running with Apache/1.3.6 on Solaris 8 (108528-15). Compiled with GCC and GNU make. Thank you for making the source available.

Al:

We exchanged notes several months ago about the possibility of Virginia Tech' Center for Applied Technologies in the Humanities (CATH) providing free hosting to ISFDB. Virginia Tech--the land-grant state university in Virginia--has been a leader in supporting online projects such as yours, and our Center has considerable computing resources available. We would like to help out in supporting the ISFDB, so please get in touch with me directly if you would like to explore the options further: Len.Hatfield@vt.edu. (Associate Director, CATH@VT).

The host I have been using for some time, Pair Networks
(information at http://pair.com/pair/compare/ ), has an account called "Advanced" for $17.95 a month that allows, among other things, custom cgi scripts, 200 MB of disk storage, 300 MB a day of transfer (9000 MB a month). It's one of eight levels of service they ofer, with the next-to-top still only $74.95 a month, and that includes over a thousand MB a day transfer.

Pair hosts over 140,000 sites; they are *not* an ISP, just a hosting service. More about them is on this page of theirs:
http://pair.com/pair/whypair.html (note that I am not a Pair employee, nor do I get commissions--I'm just a satisfied user).

Dear Al,
I'm sympathetic. My domain, <magicdragon.com>,
which includes the Ultimate Science Fiction
Web Guide, got over 10,000,000 hits in 2002.
Its costs have crept upwards over the past
7 years, as it gets a quarter Terabytes/year
file transfer. It's plain text (in HTML) and
so only about 37 Mbytes, smaller than your db,
but includes much information that just doesn't
squeeze into relational db entries. You and I
have communicated over the years. We are providing
a free service to an appreciative audience, but
the web is hostile to that philosophy today!

Johncompanies would do you a good deal since you are an Open Source project, you can get 2GB of disk space and 40GB transfers per month on your own server for $65 a month ($45 for Open Source projects).<br>
<br>
I have a number of servers with John, and I thoroughly recommend his services.<br>
<br>

Edited by author 01-25-2003 08:49 PM
I host 2 sites on wiredhub. http://www.wiredhub.net/index.html
They have very good servers and are very reasonably priced. They will, also, work with you if you need more bandwidth, etc. There top plan is $20.00 a month 1500mb of space and 20gb tranfers. If you need more than that you can talk to their sales people and they'll work something out for you.

Good luck.

Edited by author 01-25-2003 06:26 PM
I came across this while I was looking for something else (isn't that always the way?). It may be of help to you with regard to the hosting problem. The chap in charge is one of the people over at Airshare.org and seems like he could be helpful.

I've been sending out feelers of my own, to see if anyone can help with the hosting situation, but since your problem is specifically with the CPU cycles the CGI scripts are chewing up, have you considered ameliorating the problem with a caching proxy such as Squid? Searches would still be dynamic, but other database accesses would behave much more as if they were being served from static files.

Al,
     I've sent out some queries to people who may be able to help you with your bandwidth/server problems, but they need to contact you via e-mail, and I suspect the sfsite address is no linger good. Anyway, please drop me a line (lawrence@io.com). Cheers!

- Lawrence Person
http://www.io.com/~lawrence/

How many cgi's per day would you get if you registered
isfdb as a sourceforge.net project? Another option might be
to use fastcgi

Can I just say, at this terrible point in the database's and internet's history, that this has been a fantastic and highly useful service to me. Thank you.

I'm sure you've received plenty of other offers/suggestions to combat the bandwidth/space problem, but I'll add mine to the list. Have you ever heard of Burlee? They offer a hosting plan that's _supposedly_ unlimited space and unlimited bandwidth and CGI support for under $20/month. True, they have some random limits set in places, but my personal website is getting up towards a gig of storage and shows no sign of being capped. Their customer service isn't fantastic, and they're notorious for not notifying people (i.e. of renewals), but that's not a problem if you keep good records.

http://www.burlee.com/

The MIT Science Fiction Society (http://www.mit.edu/~mitsfs/) is potentially interested in hosting the ISFDB here, with a server of its own. We're already interested in running a serious database server here (replacing our more primitive setup) and using the ISFDB as a primary data source, so this isn't necessarily much hardship for us.

It may even be possible for us to have an Oracle server handle queries; not sure whether MIT's site license would allow such 'external' use.

Email mitsfs@mit.edu if you're interested and we can discuss details.

Aloha, Al:

I just read with horror about your problem with your ISP, which has resulted in termination of your CGI scripts. As a computer consultant in Hawaii, I will be happy to start the ball rolling by donating you a decent server. Please contact me after 1/27 at ken@catii.com, & you can tell me specifically what you need.

As far as a potential new ISP if you don't want to serve as one yourself, you may wish to take a look at www.cihost.net. They offer 3 GB of storage, unlimited traffic, CGI, & many other good things, all for $24.95 per month (or $240/year). I'm sure I can find other avid users of ISFDB to contribute funds towards these fees, Al, as your services are invaluable to many of us.

In the meantime, I will be happy to post the data files on my own Web-site (www.catii.com) for use until we can get you up & running -- just no updates in the meantime.

Edited by author 01-11-2003 09:32 PM
The ISFDB continues to be an immensely valuable resource -- thank you for providing it!

I have a couple of questions:

1. I've noticed that stories published at Sci Fiction have an "Originally published at Sci Fiction" note attached to them (in author bibliographies) rather than a link to an entry for the story. Is that just because each story was the only thing published in that "issue" of Sci Fiction?

2. If someone were to provide you with all the relevant data in the right format (as described in your FAQ), would you be willing to include information about stories published in other online prozines (other than the ones listed on your zines.html page)?

3. If so, how would someone go about sending you such data? I'd be happy to provide full information on the contents of a couple of magazines and various anthologies (in as close to your standard format as I can manage), if you'd be willing to incorporate it, but I saw a note in your What's New section saying that you'd turned off the ability to submit information so I wasn't sure if you'd be interested in such stuff.

I can be reached at logos@kith.org if you'd like to discuss this further.

Thanks!

The Orson Scott Card listing includes the "Ender Wiggins" series. The character's actual name is "Ender Wiggin" (no s). Also, the numbering for the series appears to be messed up.

Hope that helps.

QT - Chris Oakleaf wrote:

>I looked at your scoring for setting weights for the lists and I
>was wondering if you took into account the increased number of
>awards that can be granted today as opposed to, say, 1960?
>
>
If you go back to say, 1953, when there is only the Hugo award, there's not much you can do to boost
the values of books that weren't nominated because there was only one award at the time. The idea of
using multiple award systems to identify the top books of the year really relies on the existence of
multiple awards.

An alternative that I was exploring was the most-reviewed lists. Since print magazines have limited
space, they tend to review the most interesting books. This approach gave interesting results, but
the review of SF books prior to the early 60s tended to be fairly immature, so that method also
fails the further back in time we go.

>
>
>Give Me Liberty, listed as a novel, is an anthology.
>
>
Thanks. It will be fixed at the next update.

SF cover art from the 50's is still protected by copyright. You can see selected covers here and there associated with fair use (such as selling copies of the magazine), but it's unlikely you 'll be able to find entire collections of anyone like Bok, Emshwiller, or Bonestell.

The bibliographic summary for Liz Williams currently omits her first novel, The Ghost Sister. In the US it was published by Bantam (Spectra) in mmpb, 2001, $5.99, 0-553-58374-3; 339 pages (well, 338 numbered with an unnumbered one-page epilogue). Thanks for the time and effort that makes the ISFDb possible!

Eugene Surowitz, on the NYRSF staff, has taken on the job of updating th NYRSF Index monthly. I presume it would be helpful to you if we put you on his circulation list. It is a large Excel document.

Also, is there some arrangement we could make to use your database to allow searches of our index from our web site? The server on which Virginia Tech so graciously hosts our web site (and has for 8 years now) does not have database capabilities, and our index contains 3000+ entries.

Kathryn Cramer (kec@panix.com)

to the editor: info here has been very helpful in my research! thx! so thought I'd let you know there's an error in the posting for Jane Frank, likely a result of an error in the TofC made by the magazine, incorrectly attributing the article on San Julian Believe the author was Karen Haber. Missing however are others, here's the list from ROF and SFA mags for J.F.
Science Fiction Age Magazine
o "Digital Daredevil", art gallery feature on Fred Gambino, May 1999
o "Bio of a Space Artist" art gallery feature on Jim Burns, July 1993

Realms of Fantasy Magazine
o "Just the Right Mix of Media: Art of Daniel Horne" October 1999
o "Modern Classicism: The Gilded World of Richard Bober" art gallery feature January/February 1998
o "Northern Knights and Gothic Days" art gallery feature on Ian Miller October 1997
o "Les Edwards' Evil Twin" art gallery feature June 1995
o "Playing with a Full Deck" art gallery feature February 1996

I looked at your scoring for setting weights for the lists and I was wondering if you took into account the increased number of awards that can be granted today as opposed to, say, 1960?

The entry for Martin H. Greenberg has at least one error: Give Me Liberty, listed as a novel, is an anthology. I suspect the other forthcoming book listed as a novel is also an anthology.

Do you know if there are viewable collections of the art of 1950's SF cover artists Ed Emshwiller (Emsh)? Also interested in others including Wayne Woodard (Hannes Bok)?
Thanks,
Bill B

To the editor, thanks for posting my nonfiction publications on your website! You're missing one, however:

'A beauty, a pyhantom, and two talking heads: the psychology of confinement in SuzyMcKee Charnas' "Beauty and the Opera"' in Women of Other Worlds, edited by Helen Merrick and Tess Williams, University of Western Australia Press 1999

Use this topic to send messages to the editor of the ISFDB