My point is that the Pick OS was stable, fast, and very good at maintaining data integrity. It ultimately lost favor (to relational database systems) because typical Pick applications had all of the database logic in application code, and that logic ended up all over the place. Simple things like enforcing referential integrity, transactions, and so on. RDBMSs do that in the database, where it belongs.
MongoDB and NoSQL are not new, they are backwards steps into the pre-relational world. They seem new to anyone who doesn't know how database management was done pre-MySQL. That isn't to say that everything has to be in a relational database, or that there's never a use case for a non-relational data store like key/value or flat file, but at least recognize that those are not new ideas -- they are the prehistory of database management.
Relational Databases are great, until you have downtime.
PostgreSQL has the slowest database restore procedure the world has ever seen. So backups are basically useless for anything bigger than "trivial". Master/Slave replication is death for HA.
MySQL has Master/Master, and is actually reasonably supported "in the cloud", so that's probably the only realistic option if you don't want to run your own hardware, need HA, and don't want to pay Oracle or MS.
AFAIK Oracle and MS both have decent HA solutions, but you're probably looking at $100K starting prices, and again, you're probably going to have to run your own hardware.
My #1 database of any type would be PostgreSQL. If it was HA. And if you could run a decent sized deployment (hundreds of GB) without having to run your own hardware.
I think this more than anything is probably why NoSQL is so popular with this (HN) crowd. Because in-Memory "free" databases are a good fit for people who don't have $100K to throw at a vendor in licensing, and who don't want to lease their own hardware.
RDBMSs are way behind the curve on that. Even for people like me who genuinely like them. Unless you're willing to go with MySQL I guess. But even then, do you trust RDS to be dependable and fast? Without a dedicated DBA?
I can goto Cloudant and get a database that's very fast, has relatively modest costs, I don't have to run my own hardware, and BigCouch (derivative) has a far better HA story than any RDBMS I'm aware of outside of maybe VoltDB (which I don't have any practical experience with).
I make some compromises on ACID, which I'm learning to be OK with. I lose transactions (which really hurts some things, but rolling your own 2PC on a case-by-case basis works passably when you really need the semantics). And I have a mostly-pleasant, sometimes very fragmented set of tools to help me work with my data. It's a living.
But I'm not responsible for the hardware. It's much cheaper. And it does HA. When PostgreSQL steps up and gets that right, I'll switch back.
Not to mention, there's no excuse for the amount of tuning a decent PostgreSQL install takes. Speaking as a guy who's built several 80K-TPS systems, the barrier to entry is needlessly high. Try MS-SQL sometime and contrast. Then again, PostgreSQL is free.
Plus the free RDBMSs need enforced clustered indexes. This crontab stuff is for chumps. 99% of the time an enforced cluster would be preferable to increased insert performance for 99% of web-apps.
Good points, but I don't agree that slow database restores are "more than anything is probably why NoSQL is so popular with this (HN) crowd." The database has to be pretty big and your application has to be rather important before database restore time becomes a deciding factor. Companies with huge databases and mission-critical applications should have the resources to make their database highly available.
Every time this relational vs. NoSQL topic comes up it's clear that few, if any, of the developers experimenting with MongoDB et al. have mission-critical apps with huge databases. They didn't look at Oracle or even PostgreSQL and come to a reasoned decision to risk everything on MongoDB because it can restore from backup faster. They don't have the patience or inclination to master a relational database, they want something that appears to work right away, and they don't know the consequences down the road of abandoning ACID -- if they even know what ACID is or why they want it. And then there's the coolness factor -- no points in the cube farm for advocating some stuffy old technology.
I know there are exceptions, and I frequently use non-relational data management solutions too (though not MongoDB -- no beta testing with real data for me, thanks). But there's a difference between choosing an alternative tool because you've done the research and decided it's a reasonable solution, and choosing an alternative just to be alternative, or worse because you don't know what it's an alternative to.
I probably made that too confusing. I only mentioned the restores because the availability story for PostgreSQL is very poor and backups above the low tens-of-gigs are fairly pointless if you have SLAs to meet.
The reason why NoSQL would be popular with startups (IMO) is because it's easy to outsource the administration, they're relatively inexpensive, and they're typically much faster when faced with limited resources (both hardware and skill).
I like to think I know my way around your average RDBMS, but there's still no question that tuning your average database is non-trivial unless you're very familiar with the schema, data, production hardware and performance requirements.
I'd echo your sentiments on MongoDB, but that's my own personal prejudices. Probably just from reading about so many horror stories and what sounds like to me at least questionable implementation choices.
Four nines on a monthly SLA means you can suffer less than 5 minutes of outage without penalty. I just wouldn't bet on PostgreSQL to deliver that.
I'm not entirely convinced that a third party can either, but I think BigCouch itself fundamentally can get you there in a way that PostgreSQL can't. PostgreSQL is extremely reliable, but if you try to automate failover you could really shoot yourself in the foot and be looking at a nightmare of an outage recovery. If you go with manual failover (my choice) you're just going to have to accept that if/when you do need to failover, you're going to be able to consistently do it within minutes, but it's not going to be less than 5 minutes.
IMO HA is just one of those things that really has to be baked in at a design level.
> PostgreSQL has the slowest database restore procedure the world has ever seen. So backups are basically useless for anything bigger than "trivial". Master/Slave replication is death for HA.
What's wrong with hot standby, log shipping and heartbeat?
> Plus the free RDBMSs need enforced clustered indexes. This crontab stuff is for chumps. 99% of the time an enforced cluster would be preferable to increased insert performance for 99% of web-apps.
> What's wrong with hot standby, log shipping and heartbeat?
It doesn't work outside the lab.
I know that's audacious, but there's been plenty of Heroku based services with outages because of just this. Automated failover to standby systems is very risky and IMO just never a good idea when it comes to your database servers. You also need a bullet-proof STONITH solution so you avoid the nightmare of split-brain on your database servers then. I mean who wants to have to manually consolidate customer data after a failure?
> I don't know how easy this would be with MVCC.
I didn't consider that. Good point. Still want it. ;-)
No experience with it. It's the only real cloud option I've seen while I was investigating. It promises everything you'd want though.
As long as you can get someone else to manage it, and it's on top of some fast SSDs, it looks like a good choice to me as long as it delivers on it's promises and meets your performance goals.
MongoDB and NoSQL are not new, they are backwards steps into the pre-relational world. They seem new to anyone who doesn't know how database management was done pre-MySQL. That isn't to say that everything has to be in a relational database, or that there's never a use case for a non-relational data store like key/value or flat file, but at least recognize that those are not new ideas -- they are the prehistory of database management.