Bandwith Sucks
Apr. 1st, 2003 02:03 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
tugrikRemember that post about FM that got me the 100+ comments? Well, this might be another of those kinds of entries.
Wve' been running the Purrsia Projects since Feburary of 1996 -- just over 7 years. Wow. It's been a lot of fun, but it's also been a lot of money. Last year I tried to add up all we've spent on it and stopped before I was halfway through as the numbers were downright scary. Still, we keep going. We want the little hosting operation we have to keep running well on into the future if we can at all manage it. What's the problem, then?
What it all comes down to is Purpose. We're losing the ability to satisfy the purposes behind why we keep the sites online. Why do![[livejournal.com profile]](https://www.dreamwidth.org/img/external/lj-userinfo.gif)
revar and I run the Purrsia Projects? Well, we have a short list of goals, in no particular order. Or, rather, the order of importance changes quite often:
Some of these are easily done. The muck is pennies in the bitbucket; nothin' doin'. The netgeek projects (such as streaming radio and multiplayer game hosting) are sporadic and low-priority, so can come and go freely. I'm not a popular photographer so my photo-images sites don't get much traffic, luckily. Most of the personal friends who have sites get more traffic on their LJ than they ever do on their actual website.
The problem lies in the middle one there: giving starting webcomics creators and artists a place to grow. It's an obvious problem that's been demonstrated quite graphically during the dot-com/dot-boom era, time and time again: becoming victims of your own success. Webcomics that grow in popularity are doomed to grow past what philanthropic sites are able to give.
Bandwidth-wise, we're being crushed.
The numbers break down like this: A T1 can put out about 465GB/mo of traffic if it ran 100% all the time. Taking a theoretical saturation limit of about 85%, this means you can expect to put out around 400GB/mo if your line is blasting bits at full rate 24 hours a day. We're pushing over 320GB.
"Oh, but that means you have 80GB of bandwidth you're not using!" you might say. Not true. Remember the 'bursty' and demand-based nature of bandwidth, and the fact that you cannot save up unused bits to use later. Considering the heavy bias of North American internet users to english-written webcomics, there is a sine-wave like pattern of traffic based on the day/night cycle.
Those 'valleys' are at 1-3am or so, PST, when the bulk of our 'audience' is asleep or otherwise offline. Notice how the 'peaks' are huge upward surfaces, taking up the bulk of the time. The red line is the average, mind you. The blue is the peak; it's always pegged. When that average gets above the 1.2M line, things start grinding to a slow crawl for everybody. The only reason the muck doesn't suffer is that it's so tiny, and protected behind a tiny 'partition' the packetshaper carves out for it. But as anybody who reads The Nice and it's assorted comics we host can tell you, during the middle of the day the images load at painfully slow rates. It's getting to be as bad as Keenspace often is, which makes me grumpy.
Detailed traffic analysis proves what the problem is. The largest chunk of our link is being slurped up by a handful of long-standing comics and comic-related projects on the site. These are sites that have experienced significant growth and should have long ago been urged towards paid-providers and other higher-bandwidth sites. They're great people and we love 'em... but their sites are grinding us to a halt. If we could kick just one of the big-bandwidth boys off the island, it'd free up resources for literally dozens of startup comics that desperately need a home.
Purrsia costs us almost exactly $1000/mo in hosting and power fees per month. This does not include a single penny of the thousands of dollars we've spent on servers, UPS's, racks, network gear and assorted equipment. This does not include any money at all for the hours Revar and I spend keeping it running. Sure, we open up the donations bin, but with the notable exception of Mark Stanley's monetary contributions (he is the creator of the wonderful FreeFall comic, a site I hope to keep hosting for as long as I'm hosting anything at all on the net) the donation tin stays quite empty. As an example, we've received a whopping $50 in donations over the last 12 months, tops.
For this $12,000 a year we wanted to help people get a start. For the last half year or so now we've instead become the fiscal backers of some roleplaying bulletin boards, cross-promotion tools and other things that should have long ago been moved to a for-pay site. As much as I hated doing it, I've written and sent out the first few letters to these high-bandwidth folks asking them to please find other accomodations. We're too nice of guys to slap down a hard deadline or threaten shutting off anybody's sites, but we're definately having much less fun seeing our bandwidth dry up to a trickle.
Over the last 8 months or so we've been trying to find another way out of this. Usually the solutions involve throwing more money at the problem. As we're in an area that can't get DSL or any other more affordable bandwidth, our options are limited. We can:
A) Get another T1. Our costs nearly double. Even with the discount of buying 2 of them, we'll pay >$6000/year.
B) Go Colocated, like at Hurricane Electric. Nope... I did the math. At 320GB/mo, we'd spend far more than even having two T1's at home!
C) Get a fractional T3. Surprisingly, the per-byte costs for the ISP side are great. Unsurprisingly the local loop costs make this so expensive it's absurd.
D) Turn into a business and charge folks for hosting.
(D) sounds all nice and dandy, doesn't it? It's not that easy, though. First and foremost, the cost to the end user doesn't get acceptable until you're selling chunks of a 10MB link or better; a single T1 is just not an efficient thing to chop up and sell, price-wise. Secondly the amount of time we'd have to put into it to make it a business instead of a hobby isn't available or affordable. We're both decently paid engineers who have quite time-filling day-jobs. There's no way that a tiny hosting operation could pay one of us to work less of a day job, nevermind for both of us. Most importantly of all, however, is the big Dot Com Problem: Folks don't want to pay for something they've always had for free. The moment we start asking members to pay so-and-so $ per GB of traffic, they'll bail for Keenspot, Yahoo, Angelfire... you name it. So running a business doing small scale hosting without much of a business plan is right out, since we're not rich. If we were rich, it'd make a nice tax writeoff. :)
This leaves us with two options. Either put up with being squashed by a few 800lb bandwidth gorillas and keep paying our monthly hosting fees like good little drones, or shove a few of those gorillas off the island to get some breathing room back. We're choosing the latter, though it's unpleasant and socially awkward. I only hope the folks who've had in some cases a half-decade of free, high-quality hosting will understand when we ask them to seek other sites. We'll also give them the option of being squished into tiny spaces by our Packetshaper but I bet most of them will find that about as pleasant as eating a steak dinner through a coffee-stirrer straw.
This is definately going to affect The Nice Comics Collective, the folks at Zen Dao Miao's roleplaying forum, and a handful of other decently popular sites. I'm expecting a little fallout, but I hope it's more of the "ouch, but we understand" feelings instead of the "J00 SuXX0R!" angst. We're going to do everything we can to help folks find new homes and transition cleanly. We're open to ideas and discussion, and if anybody has a 'get out of bandwidth jail free' card we'll gladly barter for it. All the individual artist sites, personal sites, smaller/start-up webcomics, the muck, the Belfry and most all the tiny comics folks we host won't have any worries, nor will our founding site: Mark Stanley's FreeFall.
So... that being all said... Any advice from folks? Any more noble ways out of this problem that I'm not seeing right now? Believe me, if I'd struck it rich during the stock-lottery days of the Silicon Valley I'd have a full blown T3 in my garage and I'd be taking on the Keenspots and Modern Tales of the world for the top comics-hosting site crown... but as it stands, this is just how I choose to spend what most people would see as a huge car payment to just help out. I'd give more, but for now I can't.
Wve' been running the Purrsia Projects since Feburary of 1996 -- just over 7 years. Wow. It's been a lot of fun, but it's also been a lot of money. Last year I tried to add up all we've spent on it and stopped before I was halfway through as the numbers were downright scary. Still, we keep going. We want the little hosting operation we have to keep running well on into the future if we can at all manage it. What's the problem, then?
What it all comes down to is Purpose. We're losing the ability to satisfy the purposes behind why we keep the sites online. Why do
revar and I run the Purrsia Projects? Well, we have a short list of goals, in no particular order. Or, rather, the order of importance changes quite often:- We want net access at home, in the land of NoDSL. So, we got a T1.
- We are netgeeks. We want to do netgeeky things. This takes bandwidth.
- We want to help out personal friends who need small sites.
- We want to give starting webcomics creators and artists an ad-free/annoyance-free place to start.
- I personally want to host my photography business/hobby sites.
- Keeping the Muck afloat is good too.
Some of these are easily done. The muck is pennies in the bitbucket; nothin' doin'. The netgeek projects (such as streaming radio and multiplayer game hosting) are sporadic and low-priority, so can come and go freely. I'm not a popular photographer so my photo-images sites don't get much traffic, luckily. Most of the personal friends who have sites get more traffic on their LJ than they ever do on their actual website.
The problem lies in the middle one there: giving starting webcomics creators and artists a place to grow. It's an obvious problem that's been demonstrated quite graphically during the dot-com/dot-boom era, time and time again: becoming victims of your own success. Webcomics that grow in popularity are doomed to grow past what philanthropic sites are able to give.
Bandwidth-wise, we're being crushed.
The numbers break down like this: A T1 can put out about 465GB/mo of traffic if it ran 100% all the time. Taking a theoretical saturation limit of about 85%, this means you can expect to put out around 400GB/mo if your line is blasting bits at full rate 24 hours a day. We're pushing over 320GB.
"Oh, but that means you have 80GB of bandwidth you're not using!" you might say. Not true. Remember the 'bursty' and demand-based nature of bandwidth, and the fact that you cannot save up unused bits to use later. Considering the heavy bias of North American internet users to english-written webcomics, there is a sine-wave like pattern of traffic based on the day/night cycle.
Those 'valleys' are at 1-3am or so, PST, when the bulk of our 'audience' is asleep or otherwise offline. Notice how the 'peaks' are huge upward surfaces, taking up the bulk of the time. The red line is the average, mind you. The blue is the peak; it's always pegged. When that average gets above the 1.2M line, things start grinding to a slow crawl for everybody. The only reason the muck doesn't suffer is that it's so tiny, and protected behind a tiny 'partition' the packetshaper carves out for it. But as anybody who reads The Nice and it's assorted comics we host can tell you, during the middle of the day the images load at painfully slow rates. It's getting to be as bad as Keenspace often is, which makes me grumpy.
Detailed traffic analysis proves what the problem is. The largest chunk of our link is being slurped up by a handful of long-standing comics and comic-related projects on the site. These are sites that have experienced significant growth and should have long ago been urged towards paid-providers and other higher-bandwidth sites. They're great people and we love 'em... but their sites are grinding us to a halt. If we could kick just one of the big-bandwidth boys off the island, it'd free up resources for literally dozens of startup comics that desperately need a home.
Purrsia costs us almost exactly $1000/mo in hosting and power fees per month. This does not include a single penny of the thousands of dollars we've spent on servers, UPS's, racks, network gear and assorted equipment. This does not include any money at all for the hours Revar and I spend keeping it running. Sure, we open up the donations bin, but with the notable exception of Mark Stanley's monetary contributions (he is the creator of the wonderful FreeFall comic, a site I hope to keep hosting for as long as I'm hosting anything at all on the net) the donation tin stays quite empty. As an example, we've received a whopping $50 in donations over the last 12 months, tops.
For this $12,000 a year we wanted to help people get a start. For the last half year or so now we've instead become the fiscal backers of some roleplaying bulletin boards, cross-promotion tools and other things that should have long ago been moved to a for-pay site. As much as I hated doing it, I've written and sent out the first few letters to these high-bandwidth folks asking them to please find other accomodations. We're too nice of guys to slap down a hard deadline or threaten shutting off anybody's sites, but we're definately having much less fun seeing our bandwidth dry up to a trickle.
Over the last 8 months or so we've been trying to find another way out of this. Usually the solutions involve throwing more money at the problem. As we're in an area that can't get DSL or any other more affordable bandwidth, our options are limited. We can:
A) Get another T1. Our costs nearly double. Even with the discount of buying 2 of them, we'll pay >$6000/year.
B) Go Colocated, like at Hurricane Electric. Nope... I did the math. At 320GB/mo, we'd spend far more than even having two T1's at home!
C) Get a fractional T3. Surprisingly, the per-byte costs for the ISP side are great. Unsurprisingly the local loop costs make this so expensive it's absurd.
D) Turn into a business and charge folks for hosting.
(D) sounds all nice and dandy, doesn't it? It's not that easy, though. First and foremost, the cost to the end user doesn't get acceptable until you're selling chunks of a 10MB link or better; a single T1 is just not an efficient thing to chop up and sell, price-wise. Secondly the amount of time we'd have to put into it to make it a business instead of a hobby isn't available or affordable. We're both decently paid engineers who have quite time-filling day-jobs. There's no way that a tiny hosting operation could pay one of us to work less of a day job, nevermind for both of us. Most importantly of all, however, is the big Dot Com Problem: Folks don't want to pay for something they've always had for free. The moment we start asking members to pay so-and-so $ per GB of traffic, they'll bail for Keenspot, Yahoo, Angelfire... you name it. So running a business doing small scale hosting without much of a business plan is right out, since we're not rich. If we were rich, it'd make a nice tax writeoff. :)
This leaves us with two options. Either put up with being squashed by a few 800lb bandwidth gorillas and keep paying our monthly hosting fees like good little drones, or shove a few of those gorillas off the island to get some breathing room back. We're choosing the latter, though it's unpleasant and socially awkward. I only hope the folks who've had in some cases a half-decade of free, high-quality hosting will understand when we ask them to seek other sites. We'll also give them the option of being squished into tiny spaces by our Packetshaper but I bet most of them will find that about as pleasant as eating a steak dinner through a coffee-stirrer straw.
This is definately going to affect The Nice Comics Collective, the folks at Zen Dao Miao's roleplaying forum, and a handful of other decently popular sites. I'm expecting a little fallout, but I hope it's more of the "ouch, but we understand" feelings instead of the "J00 SuXX0R!" angst. We're going to do everything we can to help folks find new homes and transition cleanly. We're open to ideas and discussion, and if anybody has a 'get out of bandwidth jail free' card we'll gladly barter for it. All the individual artist sites, personal sites, smaller/start-up webcomics, the muck, the Belfry and most all the tiny comics folks we host won't have any worries, nor will our founding site: Mark Stanley's FreeFall.
So... that being all said... Any advice from folks? Any more noble ways out of this problem that I'm not seeing right now? Believe me, if I'd struck it rich during the stock-lottery days of the Silicon Valley I'd have a full blown T3 in my garage and I'd be taking on the Keenspots and Modern Tales of the world for the top comics-hosting site crown... but as it stands, this is just how I choose to spend what most people would see as a huge car payment to just help out. I'd give more, but for now I can't.
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
no subject
Date: 2003-04-01 02:20 am (UTC)So far the best answers seem to be along the line of merchandising, since advertising is pretty low these days.
no subject
Date: 2003-04-01 03:09 am (UTC)no subject
Date: 2003-04-01 03:29 am (UTC)no subject
Date: 2003-04-01 03:27 am (UTC)I'm in the middle of a pretty drastic change in direction for my own site, so I know the feeling. Personally I think I'd just say straight out that there are other things I want to do, explain the alternatives, and start throttling things right off. Even the Xeric Foundation only helps artists with their first publication, and after that it's up to them. It still makes a huge difference.
Then, of course, there's the whole s00per-technological route, inventing some improvement on bittorrent, integrating a distributed peer-to-peer download manager into the browser, and so on, but then there's the question of how much time anyone has to develop that.
no subject
Date: 2003-04-01 03:33 am (UTC)no subject
Date: 2003-04-01 05:56 am (UTC)It's been instrumental in bringing a long list of works to print, such as The King of Persia, which I pulled off my shelf last night, not expecting it'd been a beneficiary as well.
no subject
Date: 2003-04-01 09:43 am (UTC)I'm in the same boat as Tugrik from a bandwidth peespective. I live about a block from him and can't get anything better than a 56K dial-up to save my life. Often I've talked to him shooting an 802.11 connection over to his place and piggy backing through him--and forking over some cash to him in the process--but I can't justify doing that right now, not with the bandwidth saturation that he's got. It's worse than a dial-up at some points in the day from an end user experience. Horrible, to tell the truth.
I currently host a single server with Tugrik which serves up a couple of small pages, acts as my shell machine and my mail server. When I'm connected to it I regularly see latency of UP TO THREE SECONDS on key stroke echos. That's just NUTS. And as much as Tugrik is my friend one more than one occasion it has had me thinking of moving my server out from under his service and into a more expensive but responsive collocation center.
Of course, this is just my personal take on things.
no subject
Date: 2003-04-01 07:26 am (UTC)I think a lot of the major Anthro based websites are struggling moneywise now - Yerf, VCL, Flayrah, Furnation. All of them asking for donations.
We're starting to hit the point where the community may have to start to realise that although free things are neat, they're not free for the people running them, and those cool websites that people frequent so much will start to go away if the community doesn't help out.
no subject
Date: 2003-04-01 08:34 am (UTC)I'm really sorry about that. I really didn't figure the bandwidth would add up to quite so much. Isabel and I are working on making The Nice a lot more friendly right now. Thing is, we're popular enough to cost money, but not quite popular enough to make money. But we'll figure something out. Thank you very much for your generosity, both of you.
no subject
Date: 2003-04-01 05:38 pm (UTC)Let me second what Terrence said. We understand.
I have nothing but praise for Tugrik and what he's done to allow our site ("Master Zen-Dao Meow") to grow. Although nearly all of the MZDM webcomics and forum artwork is hosted on the MZDM server, the forum itself is hosted on Purrsia (as part of The Nice) and has experienced a huge surge in traffic in the past couple of years, which I'm sure contributes significantly to the bandwidth problem.
As much as I hate the thought of moving the MZDM forums from The Nice and Purrsia, I realize it's not fair to create such a disproportionate bandwidth burden. I'm looking into hosting alternatives and hope to have something by the end of the month. That may help things out enough, but as I don't have the actually Purrsia stats, I can't really say whether the removal of the MZDM forums will be enough.
Again, thanks Tugrik for all you've done.