If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!
I just thought I’d share 2 of my new favorite web sites. siteanalytics.compete.com and www.quantcast.com.
With both of these, you can enter in the address of a specific site, and they’ll give you monthly traffic estimates. Now, you have to take these numbers with a grain of salt, but I’m finding them very helpful.
Most chat applications contain an urchin that, like google analytics, stores a little piece of data for each page the user visits. If my customer get’s 10,000 hits in a day, my app gets 10,000 hits. That means I’m having to think scalability a little earlier than if I were writing an accounting application.
The results?
The # of requests per second won’t be a problem. It’s the size of the database that’s going to be the real challenge.
For the self-hosted version of ChatSpring, the solution is pretty straightforward. Cull and archive.
But for the on-demand version, things are a little trickier - Especially once I have more than 10-20 customers.
So I’m thinking about scaling the database horizontally. This would involve partitioning the database by account. Since I’m going to go with Joyent accelerators, it seems like it would be pretty cheap and easy to add/remove database servers as required.
…This is one thing that I never could have done in Django, but is pretty easy now that I’ve ported everything to PHP, and am using ADODB
So am I about to shoot myself in the foot here? If I am - Let me know. 
ps. You’re not crazy - this blog is being updated less frequently. And my ecommerce blog isn’t being updated at all. Some things have come up that have reduced my ISV time. So I’ve refocused on the most important thing - getting the product done. Strangely enough, now that the squeeze is on, My coding sessions have actually become MORE productive.
4 comments ↓
I hope my comments don’t belie my ignorance here, since I haven’t found a good way to read your historical archives in order.
But my question is, why are you using a database to house a chat log? I guess it depends on the features you are trying to incorporate, but you could just store the log itself in a file. If you design the correct file structure, it will scale much more readily than trying to write multiple chat sessions into a database. Certainly it depends on your feature set, but it’s something to consider. Realize, I’m not anti-database, I’m a dba, however if you need to scale, putting it into a database makes it much harder to begin with.
Maybe it would make sense to store certain things like login credentials, session information, the fact that there was a chat etc in a database, but then store the actual log on the filesystem.
Just and idea I thought I’d toss out there. Keep up the blogging.
Hi Doug, thanks for the comments.
I am definitely *NOT* a dba, even though I’m forced to play one in my misv.
Scaling is something that I’m concerned about, since this is really a very different kind of application than a typical CRUD system.
I’m going to really think about your flat file suggestion. Currently, it wouldn’t disrupt any functionality. But I’ll have to look at it against my roadmap.
Starr, if for some reason you decide you need a database instead of the flat file, drop me an e-mail. I can give you some pointers on making it scale. I’m also going to write up some blog posts in the next few days on good indexing technique for developers. Good indexes are a huge key to getting your database working right.
What database server are you using? Mysql has a really excellent replication system. Either that, or you could simply partition different clients into their own database so you can avoid having a single hotspot table or set of tables.
Anyway, if you have some questions, drop me a line. I’d be happy to give you some suggestions on how to make your database scale.
Hey doug,
I took you up on your offer and emailed you at the address you gave when you left your comment.
Since I hadn’t heard from you, (and since that email address looked kind of strange), I wanted to make sure you’d received it.
If, on the other hand, you’ve just been too busy to reply, don’t worry about it….
Cheers
SH
Leave a Comment