Everything I know about databases is wrong. Also, right.

June 24th, 2010, 12:48 pm PST by Greg

I have been teaching CMPT 470 for six years now, with my 13th offering going on right now. Anybody doing that is going to pick up a thing or two about web systems.

I was there for the rise of the MVC frameworks and greeted them with open arms. I watched Web 2.0 proclaim “screw it, everything is JavaScript now” and listed with suspicion, but interest. I am currently watching HTML5/CSS3 develop with excitement but wondering why nobody is asking whether IE will support any of it before the sun burns out.

There’s another thing on the horizon that is causing me great confusion: NoSQL.

The NoSQL idea is basically that relational databases (MySQL, Oracle, MSSQL, etc.) are not the best solution to every problem, and that there is a lot more to the data-storage landscape. I can get behind that.

But then, the NoSQL aficionados keep talking. “Relational databases are slow” they say. “You should never JOIN.” “Relational databases can’t scale.” These things sound suspicious. Relational databases have a long history of being very good at their job: these are big assertions that should be accompanied by equally-big evidence.

So, I’m going to try to talk some of this through. Let’s start with the non-relational database types. (I’ll stick to the ones getting a lot of NoSQL-related attention.)

Key-value stores
(e.g. Cassandra, Memcachedb) A key-value store sounds simple enough: it’s a collection of keys (that you lookup with) and each key has an associated value (which is the data you want). For Memcachedb, that’s exactly what you get: keys (strings) and values (strings/binary blobs that you interpret to your whim).

Cassandra add another layer of indirection: each “value” can itself be a dictionary of key-value pairs. So, the “value” associated with the key “ggbaker” might be {"fname":"Greg", "mi":"G", "lname":"Baker"}. Each of those sub-key-values is called a “column”. So, the record “ggbaker” has a column with name “fname” and value “Greg” (with a timestamp). Each record can have whatever set of columns are appropriate.

Document stores
(e.g. CouchDB, MongoDB) The idea here is that each “row” of your data is basically a collection of key-value pairs. For example, one record might be {"fname":"Greg", "mi":"G", "lname":"Baker"}. Some other records might be missing the middle initial, or have a phone number added: there is no fixed schema, just rows storing properties. I choose to think of this as a “collection of JSON objects that you can query” (but of course the internal data format is probably not JSON).

Mongo has a useful SQL to Mongo chart that summarizes things nicely.

(e.g. BigTable, Hbase) The big difference here seems to be that the tabular databases use a fixed schema. So, I have to declare ahead of time that I will have a “people” table and entries in there can have columns “fname”, “lname”, and “mi”. Not every column has to be filled for each row, but there’s a fixed set.

There are typically many of these “tables”, each with their own schema.

Summary: There’s a lot of similarity here. Things aren’t as different as I thought. In fact, the big common thread is certainly less-structured data (compared to the relational style of foreign keys and rigid data definition). Of course, I haven’t gotten into how you can actually query this data, but that’s a whole other thing.

Let’s see if I can summarize this (with Haskell-ish type notation, since that’s fresh in my head).

data Key,Data = String
memcacheDB :: Map Key Data
data CassandraRecord = Map Key (Data, Timestamp)
cassandraDB :: Map Key CassandraRecord

data JSON = Map Key (String | Number | … | JSON)
mongoDB,couchDB :: [JSON]

data Schema = [Key]
data BigTable = (Schema, [Map Key Data]) -- where only keys from Schema are allowed in the map
bigTableDB :: Map Key BigTable -- key here is table name

The documentation for these projects is generally somewhere between poor and non-existent: there are a lot of claims of speed and efficiency and how they are totally faster than MySQL. What’s in short supply are examples/descriptions of how to actually get things done. (For example, somewhere in my searching, I saw the phrase “for examples of usage, see the unit tests.”)

That’s a good start. Hopefully I can get back to this and say something else useful on the topic.

Computer Woes

June 10th, 2010, 10:32 am PST by Greg

My computer at home has been locking up occasionally for the last few weeks. This has been happening since my upgrade to Ubuntu 10.04/Lucid, but I suspect this is a coincidence. (1) The lockups are hard: even the SysRq magic doesn’t do anything, so I deduce that the problem is in the kernel or below. (2) I haven’t seen any reports of the new Linux kernels being flaky. (3) I tried an upgrade from the i386 to amd46 (32-bit to 64-bit) system which I had been meaning to do anyway: no change even with a significantly different kernel.

Thus, I am of the opinion that I have a hardware problem.

As a computer scientist, I don’t enjoy hardware problems, so I’m thinking about buying my way out of them. (Also, my current system is mostly 3 years old, so it’s not a crazy time to upgrade.) My current thinking:

For about $700, that would leave me with the same case and power supply (an Antec Sonata II, 450W), my video card (nVidia 7600GT, but don’t game so who cares), my Hauppauge PCI TV tuner, and my recently-upgraded hard disks.

So, the questions for the crowd: Does my “it’s hardware” assessment sound right? Is it likely that the processor/mobo/RAM swap will fix my problems? Any other suggestions for hardware purchases?

Future epic culinary journeys?

June 3rd, 2010, 11:46 pm PST by Kat

Now that me fried rice journey has ended, I find myself contemplating the next epic food journey. Having gone through one, I think I can handle another. The key is to only ask individual people, namely either my Aunt Daisy or my mom, very specific questions about whatever dish I am trying to make. I haven’t decided which dish will be next, but I thought I should write them down somewhere while they are fresh in my mind. So, here are some of the family dishes that I may want to try to make, and the person that I think knows how to make them:

My mom:
Pancit bihon, which is a Filipino rice noodle dish
Pancit sotanghon , which is more soupy than pancit bihon (I think) and made with a different type of noodle
Lumpia Shanghai (fried spring roll with meat in it)

My Ama:
Chow mein (I made this once with my Aunt Belen, so I think I can make it again, but I’m not totally sure)
Kiam pung (translation: salty rice), which is a form of fried rice that is brown (this one has soy sauce for sure!), and contains meat and sometimes hard-boiled eggs, and is flavored with spices like star anise and whole peppercorns. My Ama may have used adobo sauce from pork or chicken adobo to flavor the sauce. I think my Ama’s kiam pung is on the dry side compared to other people’s dishes (based on a Google image search of kiam pung). Also, it does not contain peanuts.
Fried meatballs

My Aunt Daisy:
Lumpia prito (fried vegetable spring rolls). She learned how to make these from my Ama. Learning from my aunt will be more straightforward. The only down-side of this dish is that it is deep fried. That’s a lot of work, and I’m not sure I’d ever really make it on my own because of that.

My Aunt Edna:
Kiam beh (translation: salty rice, but in congee-form). I loved my Ama’s kiam beh but I really don’t think I’d ever be able to learn that recipe from her. Luckily, my Aunt Edna has been making kiam beh for my Ama, and I had some last night. It’s close enough to my Ama’s (my mom’s version was good, but never quite the same), so I’m going to ask her to teach me how to make it. The linked picture of it has a recipe from a guy in Toronto. Maybe I’ll try his version and see if it tastes like my Ama’s! 🙂

The family in general:
Lumpia sariwa (fresh spring rolls). LOTS of work goes into making these. It’s more of a whole-family effort. I would just like to know the ratios of ingredients. I think that’s all I can really hope for.

That’s my list so far. Am I crazy for wanting to start what could be another soul-crushing experience?

How fried rice drove me insane (Part 3)

June 1st, 2010, 9:55 pm PST by Kat

Following Saturday’s soul-crushing family conversation about my Ama’s fried rice, during which I was informed of additonal, never-before-mentioned fried rice ingredients and cooking techniques, I have to say that I was at my wit’s end. There may have been a few instances that I sat in a ball and ricked back and forth. However, Sunday was a new day, and I was determined to take another shot at the rice on Tuesday, for Pam’s dinner night. This way I would have another set of taste buds to either confirm or reject my latest attempt.

Sunday almost killed me. We decided to start at the T&T on 1st and Renfrew (or is it Rupert – anyways, one of the R-streets). After circling both the bottom and top parking lots a couple of times without any parking luck, we decided to try somewhere else. The nearby Superstore was our next stop. Ample parking, but we couldn’t find the Wing Wing Chinese sausage anywhere. After asking a few employees, we found other Chinese sausage, but no Wing Wing. What we failed to take into account was that Wing Wing sausage was on sale last weekend, so of course there wasn’t any left! I was starting to get a little nervous. I’ll admit that conspiracy theories involving my family, The Great Canadian Superstore corporation, and the company that owns T&T were going through my head. I believe at one point I may have accused Greg of being in on it too. He pointed out that he voluntarily went to Superstore on a Sunday with me to buy the sausage. I thought that was a little suspicious, but I let it go with a warning that I was watching him. We then went to Metrotown which had both a Superstore and a T&T – ended up buying the Wing Wings at T&T (my family would be appalled that I didn’t save the 10 cents at Superstore, but I didn’t think I could get Greg to go into 2 Superstores on a Sunday). Sausages: check!

To deal with the meat drippings, I marinaded a couple pieces of chicken on Sunday night. Thank goodness I knew my Ama’s marinade recipe – again, not so much a recipe, more like mix stuff together and keep tasting it until it tastes right. Meat marinading: check!

On Monday night I roasted the chicken and collected the drippings. The meat doesn’t actually go into the fried rice – just the drippings! Anyone need two cooked chicken breasts for anything? Meat drippings: check!

Since you need cold cooked rice to make the fried rice, I also cooked the rice on Monday night so that it would have a chance to sit in the fridge for Tuesday night’s main event. Rice cooked and cooled: check.

So today was the big day. I had all of the ingredients (at least all of the ingredients that my family has chosen to tell me about). I have to admit I was a little scared to start cooking. I was seriously going to lose it if this rice tasted nothing like my Ama’s. I would be okay if it wasn’t perfect – I was just hoping to get closer than I had before. Yes, not such a lofty goal, but keep in mind, I’ve been doing this for almost 6 years – no point in getting my hopes up too high. So I started with the sausages in a large pan. I needed to get them to release their oil so that I would have oil to cook the eggs and fry the rice. They were cooking, but they weren’t giving up any oil. I was starting to panic. Pam called my Ama’s house and we were told to prick the sausage. So I made them into pincushions. After a while there was enough oil to barely cover the pan (it was a big pan though), so we pulled the sausage and I poured out the oil, leaving enough to scramble the eggs. I had a good feeling about the dish when I smelled the reserved sausage oil, and it instantly reminded me of my Ama’s kitchen. As Pam sliced the sausage and cut up the eggs, I added the rest of the oil back into the pan and added the rice. It didn’t seem like enough oil to fry the rice, but I wasn’t going to give up hope. We added the cut-up sausage and egg back into the pan, seasoned with only a little bit of salt (I think the sausage oil was pretty salty) and only 1/2 a teaspoon of sugar and…

… after almost 6 years, I, with the help of Pam and Greg, had finally done it. I made my Ama’s fried rice! The dish that has eluded me for so many years has finally been conquered. I have to admit, I did do a little dance of joy.

In the end, the ingredient that I was missing was not something you could actually see. The pivotal ingredient was the oil from the Wing Wing Chinese sausage! Aunt Daisy said that if you use a different brand it doesn’t taste the same. I don’t know whether I believe that or not, but I’m not going to go out of my way to try it. My dislike of Chinese sausage had made it previously impossible for me to make her fried rice! I tried two slices of the sausage again today – *blech* I still don’t like it. So now whenever I make my Ama’s fried rice, Pam will be receiving cooked Chinese sausages.

As for the meat drippings, I figured my Ama probably used them when she didn’t have any sausage on hand, and therefore didn’t have any sausage oil. I made a second batch of rice with vegetable oil, some of the chicken drippings, rice, egg, salt and sugar. It tasted okay, but it wasn’t THE rice. It was actually closer in flavor to my fried rice and my Ama’s.

We decided that we’re going to make Ama’s fried rice again next week for dinner night. I will also make black (seaweed) soup to go with it, which was one of Ama’s staple soups. Pam and I are very excited. We called my Ama’s house to share the good news. My Aunt Daisy was happy and my Ama was too – especially when I said that I would go over and make some for her.

And so my epic fried rice journey has come to an end. I’m annoyed that it took this long, but in a way it’s probably for the best. I now fully appreciate this fried rice and will be extremely happy every time I make it. 🙂