top bar
QuickTopic free message boards logo
Skip to Messages

TOPIC:

Fotolog Site Status

(not accepting new messages)
^     All messages            16-31 of 31  1-15 >>
31
cypherPerson was signed in when posted
11-19-2003
10:36 AM ET (US)
Ok. We're back up completely. The site is probably going to be slow for a little while while all of the servers catch up. We're monitoring them closely.

We're still trying to get a straight answer about what happened from Interland. The story they've given us so far is so outrageous and makes them look so bad that we're hoping that they find a real problem that was our fault.
30
cypherPerson was signed in when posted
11-19-2003
09:14 AM ET (US)
It appears that something happened at our hosting company (Interland) last night. We're still trying to figure out what the heck the problem is and get some straight answers from them.

We're really sorry and angry about this.

As soon as I know something, I'll share it with you all.
29
cypherPerson was signed in when posted
11-18-2003
03:20 PM ET (US)
Tonight at 3am EST we'll be running a diagnostic script on our servers to help us figure out a couple of issues that we've been working on. There's a small chance it'll make the site really slow and sucky while it's running (which is why we're doing it during the slowest hours). If so, we'll kill it in the morning before heavy traffic kicks in again.
28
cypherPerson was signed in when posted
11-12-2003
05:44 PM ET (US)
Sometimes we do indeed suck a little bit. There's been a problem for a while with the "Forgot Password" function. The page that you'd get to after you clicked the special link in your "forgot password" email, where you were supposed to enter a new password didn't say anything about the recent change in password requirements (6 characters, at least one letter and number). It also pretended to accept passwords that didn't conform to the new standards, but then wouldn't let you log in with them. Yikes.

For anyone who has been having trouble with their passwords and logging into their account, please go to the forgot password page and follow the instructions and you should be good to go.

Sorry for the confusion.
27
Fotolog1Person was signed in when posted
11-07-2003
01:37 PM ET (US)
Friday Afternoon:

We've just experienced the most stable, smooth 48 hours we've had in months. It seems like the new servers are working well, and that the new guestbook system has distributed our database load much better

Here are some of the issues that we are aware of and will work on next week:

- Guestbooks have scroll bars in certain browsers
- Guestbooks get scroll bars in certain circumstances, regardless of the browser
- Guestbooks only show first several posts in some browsers

- Changes in Gold Camera status are not reflected in guestbook username display

- Posting URLs is guestbooks (and photo captions) results in broken/partial links in some cases (usually when there is unusual punctuation in or next to the URL)

- Fotolog accounts that produce 404 page not found errors (so far we only know about ronni - if you know of other accounts like this, please let us know)
26
cypherPerson was signed in when posted
11-05-2003
09:41 AM ET (US)
Wed Morning:

- Spike installed another new database server this morning and made a few adjustments on the the new guestbook servers.

- I'm seeing a bunch of 404 errors and some failed requests on the site. Spike is investigating
25
cypherPerson was signed in when posted
11-04-2003
02:59 PM ET (US)
Fotolog status:

Servers appear to be all properly configured and running smoothly. Finally. :)

We have found the following guestbook issues:

- A few duplicate postings. We believe this only affects a couple thousand records (not many in relationship to the 8.3 million we migrated). This will NOT be fixed - if you see them in your old guestbooks and want to delete them, feel free

- URL links in GB posts are missing a space in front of them if there was originally a line break in front of them: As a fix, we are going to automatically append a space in front of them.

- URL links in GB posts open up in the GB iframe: We are going to make sure they open in their own complete window.

- Old HTML formatting (from before the time we were forced to disallow any more HTML formatting because of malicious scripting) was stripped out of old Guestbook posts: We are going to leave this as is for now. Next week we will look into the possiblity of re-migrating old guestbook postings little by little and retaining non-malicious HTML code. It's a really tricky, risky thing to do, so we might not be able to pull it off. Hackers suck.

- Some browsers are not seeing the iframe scroll bars: We're still investigating this and trying to figure out if there's any consistency to when this is happening. So far, it appears to be somewhat random - but we seem to be ok for the most common versions of the most common browsers. For now, if you can't scroll, put your cursor in the part of the guestbook you CAN see and just click and drag downwards and it should pull the rest of the guestbook up for you.

We hope to be turning on guestbook posting very soon.

Thanks for your patience everyone - this was tougher than we hoped, but things look better right now.
24
cypherPerson was signed in when posted
11-04-2003
01:19 PM ET (US)
Update: Ok, so we're not quite out of the woods on this upgrade yet. The new guestbook servers aren't playing as nicely as we had hoped with the old servers - I think we've discovered a slight incompatibility in the different versions of php that they're running. Spike is working on that now. Until then, you should expect to intermittently get an empty box where your guestbook should be.

Other issues:

We've found some issues with some of the guestbook data that was migrated. This includes double posts and, in some cases, strange formatting around URL links in the posts. We're investigating the extent of the problem. If it's REALLY REALLY bad, we'll consider re-migrating - a process that takes a few hours, unfortunately. We're hoping to avoid that, but if we DO have to go that route, that's why we didn't turn gb posting back on yet - so we wouldn't have to trash all the new postings we received today.
23
cypherPerson was signed in when posted
11-04-2003
08:10 AM ET (US)
Known issues with the new guestbook system:

- Clicking links in guestbook postings loads the new page into the guestbook frame

- Some guestbook text goes past the right edge of the guestbook frame.
22
cypherPerson was signed in when posted
11-04-2003
08:04 AM ET (US)
We're back up for a couple hours to give everything a shakedown. Then, we'll go down for a MUCH SHORTER PERIOD :) - hopefully only around an hour - to do some additional maintenance and then we'll turn everything on.

Right now we have temporarily disabled new guestbook postings so that we can verify that we migrated everything properly - so please take a look at your guestbooks.

Unfortunately, we were NOT able to bring over much older guestbook HTML formatting. We tried, but kept running into issues with old posts. Apparently, there was a lot more malicious and broken code than we expected, and it would have taken another day to check everything by hand :(

I'll be posting known issues on the site status board. Please let us know if you find anything particularly broken or strange.
21
cypherPerson was signed in when posted
11-03-2003
09:52 PM ET (US)
Slow going on the guestbook migration. We keep hitting anomalies that cause the migration scripts to poop out. We'll be working through the night on this to make sure we don't end up with missing or bad guestbook data. I don't expect us to be back up before tomorrow morning at 11am EST.

Our first full day of downtime since we launched in May of '02, but we're very optimistic the results will be worth it for everyone.
20
cypherPerson was signed in when posted
11-03-2003
03:55 PM ET (US)
Another update: The migration of the guestbooks is taking longer than we expected. You guys have posted an awful lot of messages, we're stripping out any of the old messages that were posted just to screw up the system or introduce malicious scripts, and we're being very, very careful not to lose any of your great messages.

After that process is complete, we have some data optimization to do, and then our hosting company is going to take another crack at fixing the mis-configured servers we've been running on.

At this point, expect another few hours.

Also, when the site DOES come back up, we're probably going to disable posting new guestbook messages overnight so that people can do a quick check of their guestbooks and verify that everything looks right and that we didn't miss any messages.

Thanks for your patience - we really think this upgrade will be worth the wait!
19
cypherPerson was signed in when posted
11-03-2003
02:45 PM ET (US)
Quick update:

1. It took our hosting company a lot longer than we thought to complete their part of the upgrade.

2. We've started on our end, and things are going well so far. We're hoping no more than another 2 hours at this point.

3. Apparently, we sent much too much traffic to the QuickTopic system, so we've removed the link from our site down page - hopefully that will make it easier to check in here and see what's up.
18
cypherPerson was signed in when posted
11-03-2003
10:20 AM ET (US)
We'll be taking the Fotolog site down for a couple hours today at 11:30am EST (GMT -5) to deploy our new servers, our new guestbook system and do a variety of other upgrades to the Fotolog site. We're hoping the site will only be down for 2-3 hours.

Check in here for updates (if there's anything interesting or unusual that comes up during the process).

And, thanks to everyone who helped us test out the new guestbook system last night. We think we took care of MOST of the issues - but, as with any new technology, we'll probably have some kinks to work out after we deploy - so your help and patience will come in handy again :)
17
cypherPerson was signed in when posted
10-30-2003
09:57 PM ET (US)
Also, you may have noticed this - every half-hour (on the hour and half-hour) the site hiccups for a minute. We're trying to identify the process that's causing this and hopefully improve it so it doesn't do this.

In the meantime, if you have a problem posting or viewing EXACTLY at 10:00am (for instance), that's probably what it is.
16
cypherPerson was signed in when posted
10-30-2003
09:54 PM ET (US)
Quick update:

1. We are working on getting the new servers ready to launch. In the meantime, expect that the syncing problem between the master and slave databases will continue. As one gets further and further behind the other, depending one which one you get routed to when you view a Fotolog page, you might be getting the most up to date version of the page or an older one. During lulls in activity, things catch up.

2. From around 8:30 to 9pm tonight, we had a glitch on one of the my. servers which may have prevented you from posting messages or photos. It's ok now (and not indicative of anything - just one of those periodic server chokes)

3. Spike has created and installed some nifty server monitoring tools that notify us when there's a problem and automatically restart the webservers if they choke throughout the day (as webservers sometimes do), so hopefully we won't have any more of those days where something is down for a while before we realize it.

4. We're a little (or a lot) behind on Admin mail - we're working hard to catch up. If you wrote in, hang in there - we WILL answer you.
^     All messages            16-31 of 31  1-15 >>