I'd say it's more analogous to SimpleDB than S3 -- it's a hosted database. Although it is possible to store very large objects/files in MongoDB so I suppose it is an alternative to both.
It's easy to run MongoDB on EC2 also, so that is an option too if one uses it. (Of course one could connect to a mongohq instance from ec2 over the internet too.)
Ryan Graves
· 2 months ago
Correct, it is analogous to Amazons SimpleDB.
Ryan Graves
· 2 months ago
This post got me curious to learn more about this DB structure so I found an awesome post that explains document database structures and BigTable, Google's use of this structure for their obviously superior scalability.
Thanks for bringing this up Fred, it's always fun to dive in an learn a little bit more.
Nik
· 2 months ago
Hi Fred,
We use S3 for our document storage and the Key Value Pair architecture really works for us. Is MongoHQ primarily going up against S3?
I am curious to see MongoHQ differs from S3?
A key suggestion to differentiate MongoHQ from S3 is to provide value add on the document/file itself. For e.g. If its a word/pdf document, can one automatically convert it into a flash player embedded document so that the web service doesn't have to do the additional work of converting and showing it on the website. In other words, for every document stored in the DB, can MongoDB provide an easy web representation of the document so that its not just a regular DB but it actually works extra hard to make your documents/files web "compatible".
So, if the the NYTimes, has documents, videos, files etc, you not only store the files for them in a scalable fashion but you also do the additional work for it to easily show it on the web.
Maybe, a stretch analogy could be a Twilio for document/file storage.
Nik
Kevin Marshall
· 2 months ago
Without knowing much of anything, I believe that MongoDB, especially a hosted version, is much more like the combination of Amazon S3 and Amazon SimpleDB (both the database-like features as well as the actual large data store)...S3 is really just about data storage, while SimpleDB is about giving you some database-like features from a key/value system...
Awesome stuff though on all fronts for us techies to learn and play with! ;-)
Mark Essel
· 2 months ago
I need to familiarize myself with setting up a nice scalable database. I'd rather use a system that handles most of the difficult work of keeping it fast no matter how much it bloats.
Kevin Marshall
· 2 months ago
Most all of these key/value type databases (Mongo, SimpleDB, Google's Big table) are designed to solve just that pain point...right now the real question in picking one over another seems to be what company/environment you want to be tied to (ie. Google, Amazon, or your own hosting world)...and this directly relates to all the other tech. decisions you'll be making (like what language you want to work in, what your tech. budget long term is, what type of scale you expect to have to support, etc.)...
Mark Essel
· 2 months ago
Appreciate your shared experience in this area Kevin.
Mark Essel
· 2 months ago
That sounds pretty nifty Nik. How do you like using S3 so far? How simple to get into was it, compared to something like Googles datastore?
Nik
· 2 months ago
Actually, its quite to use S3. We didn't try Google's data store. But we did try and see if we should install our document storage DB (some of the players are mentioned here- http://www.metabrew.com/article/anti-rdbms-a-li... ) but ultimately decided to go with S3 because of simplicity, familiarity and scalability...
But, our documents don't live in the ether, the documents we store (like quite a few other web services) need to be shown on our service and "hacking" a web compatible format for all the various document formats is not fun...
Mark Essel
· 2 months ago
I shall explore their database API, but my limited expertise in this area will be a quick judge (if it's too tricky I'm skipping it). I want ultra simple kindergarten building blocks for my web apps :)
fredwilson
· 2 months ago
that is good feedback
Mark Essel
· 2 months ago
I wasn't able to sign up, seems I need a special code? edit: nevermind, found the request beta code link
Brian
· 2 months ago
We're really interesting in MongoDB but have been waiting for the auto sharding capabilities to be ready for production. Have these limitations been overcome in the hosted version from 10gen?
im2b_dl
· 2 months ago
Fred I really think I want to talk about this... ; ) ...a challenger to S3 without the pressure of retail going through Amazon...yeah I really want to talk about this.
daryn
· 2 months ago
How do S3 (or any of the amazon web services) put pressure on retail going through Amazon? They are backend platforms that have nothing to do with retail, and you could easily build a website using s3, sqs, simpledb, and ec2, but use paypal for your payments layer instead of amazon payments. You could even build your own internet bookstore on AWS if you wanted to...
What I like about the 10gen stuff is that you can use their sdk to develop scalable software, but potentially deploy it anywhere, whether on 10gen's cloud, some other hosting provider, or on your own hardware.
At a glance Mongo looks like a CouchDB / a hybrid of SimpleDB and S3, but I haven't needed to really dig in to any of them yet.
im2b_dl
· 2 months ago
We are a different entity. ...that Amazon would like to see our retail/referral go through them.
daryn
· 2 months ago
I don't understand what you're saying, but all of amazon's services are decoupled: you can pick and choose and use any or all of them.
Here's an example of a 3rd party service that someone wrote to let you sell digital goods using S3 as your document store, and paypal as your payment gateway. Amazon doesn't discourage this, even though it could be done entirely on their platform. In fact, it is one of the featured services in the AWS showcase.
of course we can decouple it. We are a different situation. sorry
BillSeitz
· 2 months ago
I'd have concerns about the overhead of every db hit from my app-server going over the public net.
ehwizard
· 2 months ago
Yes - going over the public net would be an issue. I think the best use case is in the same data center as the hosting service. Eventually, MongoHQ should have servers at the big hosting companies, so you just use the servers that are local to you.
Julien
· 2 months ago
I think there is a huge trend for "on-demand" services, related to hosting. Of course S3 and EC2 set the trend, but we now how MongoHQ for databases, we have Superfeedr for RSS feed parsing on demand... etc.
Diego Sana
· 2 months ago
I discovered mongodb in a post you did here back in february and instantly fell in love with the concept. I've been using it in an under development app and i'm very pleased with the features and performance. Their mailing list (a google group) is very active and the founders/employees promptly answers all sorts of questions, and judging by myself and all the excitement i see there by people discovering mongo, i think you did a very nice investment on this one that will definitely pay off :)
I do host mongodb on my own dedicated server, but i think that the hosted offer on mongoqh is an excelent idea as many developers don't want/don't know/can't afford a dedicated hosted and usually go with a shared hosting where mysql is usually their only database option.
fredwilson
· 2 months ago
that is great feedback. thanks so much.
Ryan Graves
· 2 months ago
I agree with Mark Essel, a simpler explanation would do them a lot of good.
From my limited understanding I totally thought this was an Amazon SimpleDB comparison (what we use on our startup, heavy with images), but on their site they have product comparisons and they don't compare themselves to Amazong SimpleDB.
?
ShanaC
· 2 months ago
So all this talk of databases is above my head. And I think that's silly when it comes to the market of databases.
I was talking to family friends very early this summer, at the first mention of the whole NoSQL movement. Not that I fully understand it. This family friend had used SQL and was looking for a change in life, she was in her late 40s-early 50s, and was looking to say try to learn SAP in some mysterious way. NoSQL hadn't even hit her radar, and it still really isn't, despite my insistence that it will.
One of the reasons is that the literature if you are just starting out is way above most people's heads. Aim to explain everything. At the most basic level. You'll capture a market no one else has yet (I mean, why would anyone else go for that market, we're just not that cool...)
kenberger
· 2 months ago
Cool- I finally get to put on my East Agile hat on this blog without sounding pitchy :)
East Agile is an elite engineering team doing contract web and mobile development. We’d be happy to work with such an offering and off-load the DB heavy lifting that’s not really our core competency-- this comes up more and more as we work with clients' more robust and more spikey apps.
Hosted MongoDB's value prop seems similar to that of Engine Yard, who is probably the best solution out there for Ruby deployment and management (we are an EY development partner).
As it relates to startups' adoption, key to this is pricing model: Engine Yard just changed their pricing for smaller companies so the cost can start small and scale up if and when usage grows.
We really like mongodb, and actually use in a key piece of our infra. However, I'll be extremely surprised if the hosted offering will be used for anything serious. It's not only the networking latencies, cost and indeterministic performance. The main issue is that working in an architecture that is outsourced horizontally is extremely hard. Counter-intuitively as mongodb becomes more mature and robust, it becomes easier to deploy it and thus the incentive to use it in a saas model goes down.
What will work, however, is to create something like amazon images that provide value and are vertical specific, like cloudera is doing successfuly. a simple example will be a great geo based db that provides a complete location awareness solution.
Incidentally, one of the cool features of MongoDB is that it's open source. For example, today we found a bug in mongodb's handling of long vars somewhere by being able to dive into the code. Will be hard doing that in a saas model :)
Mitch Pirtle
· 2 months ago
I think this is a great way to demonstrate the simplicity and power of Mongo. I was giving sessions at a JoomlaDay event in NYC on Monday, and when I described Mongo I got a lot of quizzical stares...
I whipped out my laptop and showed them the Mongo Shell, and suddenly lightbulbs were going off everywhere in the room.
This hosted service is going to be a great way for developers that are unsure of the value proposition of Mongo to kick the tires and see for themselves what Mongo can do.
fredwilson
· 2 months ago
Exactly. That's why I blogged about it.
Laurent
· 2 months ago
Hi Fred,
We've been fairly involved with 10gen and the latest builds of MongoDB. It has been a key part of our elastic infrastructure since we launched a few weeks ago, fitting perfectly between the other providers that we use (namely MySQL and S3). Scaling smoothly, Silentale currently stores more than 8 million large items ("documents" for MongoDB) and grows by about 4% per day. The 10gen team has been absolutely fantastic in supporting us and moving forward with their roadmap. I really hope that the hosted version will stimulate more companies to use and contribute to MongoDB in production systems.
Laurent
fredwilson
· 2 months ago
Wow. Thanks for sharing this. It is so great to hear stories like this
hunternield
· 2 months ago
I've been using MongoDB for a few projects recently. It's an incredible system for rapid web development and has proved very solid for serious production work. We've even managed to sneak it into a deployment for the local arm of a large Japanese automaker.
When I heard about MongoHQ I also signed up for the beta. As expected being an early release, a few rough edges but could be a great option in connection with other EC2 cloud solutions. As a test we've deployed an early prototype of a new app we're building. Very smooth going, if they can nail it with the price then it will be very appealing to go alongside services like Heroku or plain old EC2.
fredwilson
· 2 months ago
That is great to hear hunter. We are hearing more and more good things about mongo
dmerriman
· 2 months ago
That's great to hear. By any chance could you blog about your usage of the product? We are getting a lot of questions about how people are using mongodb in the real world.
vpras_krish
· 2 months ago
We’re doing a webinar on MongoDB on Oct 30. It’ll be an overview of MongoDB, and will also have Ian White from Business Insider talking about how they are using MongoDB in production. This should answer some of the questions people posted here on MongoDB vs other databases, understanding production use cases etc.
If you have ideas for future webinars, which could be to share an interesting way in which you are using MongoDB or to learn more about a specific aspect of MongoDB, we'd love to hear. Please share your thoughts in the comments or email us at info@10gen.com.
All Comments
It's easy to run MongoDB on EC2 also, so that is an option too if one uses it. (Of course one could connect to a mongohq instance from ec2 over the internet too.)
For those wanting a simple but helpful explanation check out:
http://carsonified.com/blog/dev/should-you-go-b...
...from Carsonified. That guy's a stud.
Thanks for bringing this up Fred, it's always fun to dive in an learn a little bit more.
We use S3 for our document storage and the Key Value Pair architecture really works for us. Is MongoHQ primarily going up against S3?
I am curious to see MongoHQ differs from S3?
A key suggestion to differentiate MongoHQ from S3 is to provide value add on the document/file itself. For e.g. If its a word/pdf document, can one automatically convert it into a flash player embedded document so that the web service doesn't have to do the additional work of converting and showing it on the website. In other words, for every document stored in the DB, can MongoDB provide an easy web representation of the document so that its not just a regular DB but it actually works extra hard to make your documents/files web "compatible".
So, if the the NYTimes, has documents, videos, files etc, you not only store the files for them in a scalable fashion but you also do the additional work for it to easily show it on the web.
Maybe, a stretch analogy could be a Twilio for document/file storage.
Nik
Awesome stuff though on all fronts for us techies to learn and play with! ;-)
But, our documents don't live in the ether, the documents we store (like quite a few other web services) need to be shown on our service and "hacking" a web compatible format for all the various document formats is not fun...
edit: nevermind, found the request beta code link
I really think I want to talk about this... ; ) ...a challenger to S3 without the pressure of retail going through Amazon...yeah I really want to talk about this.
What I like about the 10gen stuff is that you can use their sdk to develop scalable software, but potentially deploy it anywhere, whether on 10gen's cloud, some other hosting provider, or on your own hardware.
At a glance Mongo looks like a CouchDB / a hybrid of SimpleDB and S3, but I haven't needed to really dig in to any of them yet.
decoupled: you can pick and choose and use any or all of them.
Here's an example of a 3rd party service that someone wrote to let you sell
digital goods using S3 as your document store, and paypal as your payment
gateway. Amazon doesn't discourage this, even though it could be done
entirely on their platform. In fact, it is one of the featured services in
the AWS showcase.
http://www.digitalgoodsstore.com/
I do host mongodb on my own dedicated server, but i think that the hosted offer on mongoqh is an excelent idea as many developers don't want/don't know/can't afford a dedicated hosted and usually go with a shared hosting where mysql is usually their only database option.
From my limited understanding I totally thought this was an Amazon SimpleDB comparison (what we use on our startup, heavy with images), but on their site they have product comparisons and they don't compare themselves to Amazong SimpleDB.
?
I was talking to family friends very early this summer, at the first mention of the whole NoSQL movement. Not that I fully understand it. This family friend had used SQL and was looking for a change in life, she was in her late 40s-early 50s, and was looking to say try to learn SAP in some mysterious way. NoSQL hadn't even hit her radar, and it still really isn't, despite my insistence that it will.
One of the reasons is that the literature if you are just starting out is way above most people's heads. Aim to explain everything. At the most basic level. You'll capture a market no one else has yet (I mean, why would anyone else go for that market, we're just not that cool...)
East Agile is an elite engineering team doing contract web and mobile development. We’d be happy to work with such an offering and off-load the DB heavy lifting that’s not really our core competency-- this comes up more and more as we work with clients' more robust and more spikey apps.
Hosted MongoDB's value prop seems similar to that of Engine Yard, who is probably the best solution out there for Ruby deployment and management (we are an EY development partner).
As it relates to startups' adoption, key to this is pricing model: Engine Yard just changed their pricing for smaller companies so the cost can start small and scale up if and when usage grows.
(We'll let you know how our Mongo usage goes.)
http://eastagile.com
What will work, however, is to create something like amazon images that provide value and are vertical specific, like cloudera is doing successfuly. a simple example will be a great geo based db that provides a complete location awareness solution.
Incidentally, one of the cool features of MongoDB is that it's open source. For example, today we found a bug in mongodb's handling of long vars somewhere by being able to dive into the code. Will be hard doing that in a saas model :)
I whipped out my laptop and showed them the Mongo Shell, and suddenly lightbulbs were going off everywhere in the room.
This hosted service is going to be a great way for developers that are unsure of the value proposition of Mongo to kick the tires and see for themselves what Mongo can do.
We've been fairly involved with 10gen and the latest builds of MongoDB. It has been a key part of our elastic infrastructure since we launched a few weeks ago, fitting perfectly between the other providers that we use (namely MySQL and S3). Scaling smoothly, Silentale currently stores more than 8 million large items ("documents" for MongoDB) and grows by about 4% per day. The 10gen team has been absolutely fantastic in supporting us and moving forward with their roadmap. I really hope that the hosted version will stimulate more companies to use and contribute to MongoDB in production systems.
Laurent
When I heard about MongoHQ I also signed up for the beta. As expected being an early release, a few rough edges but could be a great option in connection with other EC2 cloud solutions. As a test we've deployed an early prototype of a new app we're building. Very smooth going, if they can nail it with the price then it will be very appealing to go alongside services like Heroku or plain old EC2.
Details & registration at: http://mongodb1.eventbrite.com/
If you have ideas for future webinars, which could be to share an interesting way in which you are using MongoDB or to learn more about a specific aspect of MongoDB, we'd love to hear. Please share your thoughts in the comments or email us at info@10gen.com.