Amazon SimpleDB

From IronPython Cookbook

Revision as of 09:32, 15 October 2010 by MichaelFoord (Talk | contribs)
(diff) ← Older revision | Current revision (diff) | Newer revision → (diff)

This page is adapted from a page describing how to import Amazon SimpleDB stuff into Resolver One.

Before you get started, you should know that SimpleDB is currently in Limited Beta - before you put any time and effort into trying to work with it, you should sign up with Amazon (if you don't already have an AWS account), join the beta, and wait until they give you access.

About Amazon SimpleDB

Amazon SimpleDB is a Web Service provided by Amazon which, just as you would expect, lets you store data on their servers like you would with a database. For full details about the service and how to use it, you should look at their website. However, here's the short version.

Amazon SimpleDB is a cut-down database system, and as such it structures data quite differently to a normal relational database - in a manner that is actually quite similar to a spreadsheet. Each database is structured into "domains", which can be regarded as similar to worksheets. A query to the database can only access a single domain (unlike relational database queries, which can "join" multiple tables). However, the domain is quite freely structured. It is made up of a number of items (which you can see as being similar to rows in the worksheet), each item having a number of attributes (which you can see as being similar to columns). There is no requirement for each row to have the same set of attributes - in relational database terms, it is highly denormalised.

In Amazon's view, this kind of simpler and less structured kind of data store "is easy to use and provides the core functionality of a database - real-time lookup and simple querying of structured data - without the operational complexity."

Getting the client-side libraries

While you could, in theory, write your own web service access code to talk to the Amazon AWS systems directly, it is much easier to use something someone else has already written. Amazon have provided a C# library for accessing Amazon SimpleDB, so this page describes how to use it as a .NET library from within IronPython. (An independent group are working on a Python library, which we will be looking at later.)

Here's a step-by-step guide to creating the DLL that you need:

  • If you don't have a version of Visual C# installed, you should get the free "Express Edition" from Microsoft's website.
  • Next, you can download the Amazon SimpleDB C# library from Amazon's website.
  • Uncompress the C# library, and from Visual C# open the sample project amazon-simpledb-2007-11-07-cs-library\src\Amazon.SimpleDB.sln.
  • Compile it (hit F6 or select "Build solution" from the "Build" menu).
  • Take the file Amazon.SimpleDB.dll from amazon-simpledb-2007-11-07-cs-library\src\Amazon.SimpleDB\bin\Release and put it somewhere safe where you can use it later. This DLL file contains the client access code for SimpleDB.

Using the DLL from IronPython

Firstly, start off an ipy console in the directory that contains the DLL:

IronPython 1.1.1 (1.1.1) on .NET 2.0.50727.312
Copyright (c) Microsoft Corporation. All rights reserved.

You need to load up the DLL...

>>> import clr
>>> clr.AddReference("Amazon.SimpleDB")

...and then connect to it. (Obviously you need to replace Access key and Secret key below with the API keys Amazon gave you.)

>>> from Amazon.SimpleDB import AmazonSimpleDBClient
>>> service = AmazonSimpleDBClient("Access key", "Secret key")

Now, you can create a domain:

>>> from Amazon.SimpleDB.Model import CreateDomain
>>> createDomainAction = CreateDomain().WithDomainName("MyStore")
>>> response = service.CreateDomain(createDomainAction)

(I think you can pretty much ignore the response for this call, as it will raise an exception if there are any errors.) can put some data into it:

>>> from System.Collections.Generic import List
>>> from Amazon.SimpleDB.Model import PutAttributes
>>> from Amazon.SimpleDB.Model import ReplaceableAttribute
>>> def AddItem(domainName, itemName, attributes):
...     putAttributesAction = PutAttributes().WithDomainName(domainName).WithItemName(itemName)
...     convertedAttributes = List[ReplaceableAttribute]()
...     for name, value in attributes:
...         convertedAttributes.Add(ReplaceableAttribute().WithName(name).WithValue(value))
...     putAttributesAction.Attribute = convertedAttributes
...     service.PutAttributes(putAttributesAction)
>>> AddItem("MyStore", "Item_01", [("Category", "Clothes"), ("Subcategory", "Sweater"), ("Name", "Cathair Sweater"), ("Color", "Siamese"), ("Size", "Small"), ("Size", "Medium"), ("Size", "Large")])
>>> AddItem("MyStore", "Item_02", [("Category", "Clothes"), ("Subcategory", "Pants"), ("Name", "Designer Jeans"), ("Color", "Paisley Acid Wash"), ("Size", "30x32"), ("Size", "32x32"), ("Size", "32x34")])
>>> AddItem("MyStore", "Item_03", [("Category", "Clothes"), ("Subcategory", "Pants"), ("Name", "Sweatpants"), ("Color", "Blue"), ("Color", "Yellow"), ("Color", "Pink"), ("Size", "Large"), ("Year", "2006"), ("Year", "2007")])
>>> AddItem("MyStore", "Item_04", [("Category", "Car Parts"), ("Subcategory", "Engine"), ("Name", "Turbos"), ("Make", "Audi"), ("Model", "S4"), ("Year", "2000"), ("Year", "2001"), ("Year", "2002")])
>>> AddItem("MyStore", "Item_05", [("Category", "Car Parts"), ("Subcategory", "Emissions"), ("Name", "O2 Sensor"), ("Make", "Audi"), ("Model", "S4"), ("Year", "2000"), ("Year", "2001"), ("Year", "2002")])

...and read it out again:

>>> from Amazon.SimpleDB.Model import Query, GetAttributes
>>> queryAction = Query().WithDomainName("MyStore")
>>> response = service.Query(queryAction)
>>> if response.IsSetQueryResult():
...     for itemName in response.QueryResult.ItemName:
...        print itemName
...        getAttributesAction = GetAttributes().WithDomainName("MyStore").WithItemName(itemName)
...        response = service.GetAttributes(getAttributesAction)
...        if response.IsSetGetAttributesResult():
...            for attribute in response.GetAttributesResult.Attribute:
...                print "    (%s, %s)" % (attribute.Name, attribute.Value)
    (Color, Paisley Acid Wash)
    (Size, , 32x34)
    (Size, 30x32)
    (Size, 32x32)
    (Size, 32x34)
    (Category, Clothes)
    (Subcategory, Pants)
    (Name, Designer Jeans)
    (Year, 2006)
    (Year, 2007)
    (Color, Blue)
    (Color, Pink)
    (Color, Yellow)
    (Size, Large)
    (Category, Clothes)
    (Subcategory, Pants)
    (Name, Sweatpants)
    (Year, 2000)
    (Year, 2001)
    (Year, 2002)
    (Model, S4)
    (Make, Audi)
    (Category, Car Parts)
    (Subcategory, Emissions)
    (Name, O2 Sensor)
    (Color, Siamese)
    (Size, Large)
    (Size, Medium)
    (Size, Small)
    (Category, Clothes)
    (Subcategory, Sweater)
    (Name, Cathair Sweater)
    (Year, 2000)
    (Year, 2001)
    (Year, 2002)
    (Model, S4)
    (Make, Audi)
    (Category, Car Parts)
    (Subcategory, Engine)
    (Name, Turbos)

And that should be enough to get you started :-) The rest of the API is pretty easy to follow, and you can find out about it from the developer documentation here.