Tue Nov 16 2010

This past weekend I was able to attended and present at the Chippewa Valley Code Camp.  This was the was my first time to CVCC.  Just a great event and loads of great people there.  I know it's no easy feat to pull off a free community event so a huge thanks to both Doug and Dan for all of their hard work.  I also thought it was just awesome to see all of the students who attended. 


Ok, OData.  My session looked a little something like this:

OData, it's like ATOM but with some extra stuff > You know Open Data Protocol or OData. It's a Web protocol for querying and updating data that provides a way to unlock your data and free it from silos that exist in applications today. OData does this by applying and building upon Web technologies such as HTTP, Atom PublishingProtocol (AtomPub) and JSON to provide access to information from a variety of applications, services, and stores.

We are going to explore how to publish and consume an OData services and have a little fun along the way.

So what is OData?  Well OData.org defines it as such:

The Open Data Protocol (OData) is a Webprotocol for querying and updating data that provides a way to unlock your data and free it from silos that exist in applications today. OData does this by applying and building upon Web technologies such as HTTP, Atom PublishingProtocol (AtomPub) and JSON to provide access to information from a variety of applications,services, and stores. The protocol emerged from experiences implementing AtomPub clients and servers in a variety of products over the past several years.  OData is being used to expose and access information from a variety of sources including, but not limited to, relational databases, file systems, content management systems and traditional Web sites.

I find myself talking about OData a lot.  Why?  Well interesting enough it's not because of some hidden agenda, I'm just drawn to it.  Every application I have been part of has had to deal with data in some fashion.  I realize this isn't the case for every application, it's just my experience. For me, I think the draw comes down to this:

  • Its simplicity I like REST I like that in .NET it's built on WCF I like the overall industry support around the protocol It takes what works very well, http and expands on existing success* I like how you can easily use it with things like JQuery

But enough about me...  There are really two parts of OData or for that matter any web service really. Consuming and Producing or the client side vs. the server side.  Let's start with consuming some OData.


For the purposes of consumption, I am going to use Fiddler, Internet Explorer 9 and StackOverflow and my own demo for these examples.  You can find StackOverflow's OData services at http://odata.stackexchange.com/.  Lets walk through a few queries against StackOverflow.  Another thing to point out.  Since OData is built on ATOM when you query in the browser you will be show the Feed Reading view.  You can actually disable that by going to Tools -> Internet Options -> Content -> Feeds and WebSlices Settings -> Uncheck Turn on feed reading view.




What Entities are available to consume? http://odata.stackexchange.com/stackoverflow/atom/.  The result ( below ) will show you a couple of things.  First it will show you the entities you can actually query ( highlighted ), and secondly the url of where that that entity exists ( the href attribute for the collection element ).

<?xml version="1.0" encoding="utf-8" standalone="yes"?>









Next lets query the base service to find out what technical goo we can gleam.  To do so I simple ask the service for $metadata http://odata.stackexchange.com/stackoverflow/atom/$metadata. This returns all of the geeky details you were looking for in regards to those exposed entities.  At this point you could also use the OData Visualizer in Visual Studio to get that graphical look of the service.  After you add your service reference, just right click on it and select View in Diagram.




Querying an entity is as simple as navigating to the url that was provided to you.  Let's query for the Posts: http://odata.stackexchange.com/stackoverflow/atom/PostsNote the url is case sensitive so posts will not work. One of my favorite parts about OData is the ability to just query it. Think about the web services of yester year.  As the business changed we would be asked to refactor our services.  Maybe we started with something simple like this:

public string GetCustomersByState( string state )

Things were great, and a few months later someone wanted Active Customers by state.

public string GetActiveCustomerByState ( string state )

Not a big deal right?  Hell, we were most likely even calling into the same business services and just added some flag to now query all or by "status".  Of course now, the business came back and said.  I want to query customers by state and by status.  Now you might be thinking:

public string GetCustomers ( string state, string status )

But in fact you're really just frustrated. Maybe you could have been more creative with your services? Maybe more abstract? Of course now you have services out there that will be a huge deal to change. I mean it's in production and billions of customers are using it right?  I have been there. Ok maybe not billions of customers but all it really takes it two to cause headaches.  OData actually address this very scenario head on.  You can now query your endpoint without the need of having predetermined operations. BOOYA!! Don't get me wrong, you will most likely still need some operations maybe even a lot, but what about that scenario above?  As developers we are horrible at estimating.  As humans it turns out we are just horrible at predicting the future.  You could never predict the future of your web services or the users who want to consume them.  Of course you still want agility and I am sure you had to have them last week, right.  So let's query StackOverflow.

If this http://odata.stackexchange.com/stackoverflow/atom/Posts gives us all the posts. How can we get just Posts that are tagged with css? Or what if we wanted to know how many posts there were? Or the count on Posts tagged css? I am sure the ninjas at StackOverflow are worried about all the questions I have about their data.

That is pretty sick. 


But now I want JSON. Simple. Now it might be different depending on the libraries your using. In .NET if you want JSON returned you modify your http header to include: accept: application/json.


Host: odata.stackexchange.com
accept: application/json

You can find more about JSON support in odata here: http://www.odata.org/developers/protocols/json-format.  After reading that you might also notice the format parameter $format=json.  At the time of writing this, that parameter isn't supported out of the box in .NET 4. You have to change the accept header ( for out of box support ) or write some server side logic to support the parameter.


But not all of my users are geeks.  Good! Lets use Excel.  Yea I said it.  As it turns out Excel is actually a great place to consume OData services too. There is an add on to Excel called PowerPivot




Once you select _From Data Feeds.  _You will be prompted with a dialog box asking you for the URL.  After it inspects the endpoint it will give you the option of which entities from the service you want to import.  After you've selected things it will create a worksheet per entity populated with data.  From there you can get our your Excel ninja skills and have some fun.


What about posting to the service?  Posting to the services isn't very complicated.  You will need to create an instance of _DataServiceContext _found in _System.Data.Services.Client. _Once we create an instance of the context we set our merge options.  Then we will need to create and populate the model.  Since we used Visual Studio to add a service reference, it automatically created objects bases on the entities found from the service.  One we have added the data to the object, we add it to the context and then save it.  The save will result in posting it to the service.  Here is a sample code snippet from the demo rather than StackOverflow.

DataServiceContext context = new DataServiceContext (new Uri(@"http://localhost:9998/Services/Reporting.svc/"));

context.MergeOption = MergeOption.AppendOnly;

Evangelist evangelist = new Evangelist();

evangelist.Name = "Clark Sell";

evangelist.District = "Midwest";

evangelist.State = "IL";

context.AddObject("Evangelists", evangelist);


In a later post I will explore posting more complicated data structures.


Up to now we have seen how to query a service. Here is the funny part, that might be actually harder than creating the service itself.  Let's start with just exposing an Entity Framework model as an OData endpoint.  First we will need to create the Service itself.  Add New Item, and Select a WCF Data Service.  The result of that is your service:

public class blog : DataService< /* TODO: put your data source class name here */ >


                // This method is called only once to initialize service-wide policies.

                public static void InitializeService(DataServiceConfiguration config)


                        // TODO: set rules to indicate which entity sets and service operations are visible, updatable, etc.

                        // Examples:

                        // config.SetEntitySetAccessRule("MyEntityset", EntitySetRights.AllRead);

                        // config.SetServiceOperationAccessRule("MyServiceOperation", ServiceOperationRights.All);

                        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;



Notice our service inherits from DataService. That DataService also takes a generic which happens to either be our Entity Framework model or our own class. During InitializeService we have the change to setup things. This is where we can configure operations, entity rights and so on. You can set Entity Access on all '*' or each entity. You also set what can be done to it EntitySetRights. The rights are detailed here: http://msdn.microsoft.com/en-us/library/system.data.services.entitysetrights.aspx.


Next, Operations.  If you need one, you have to tell the DataService you have one.  The SetServiceOperationAccessRule is where you do so.  Let's take my demo example. 


    config.SetServiceOperationAccessRule("SomeOperation", ServiceOperationRights.All);


Now for our actual operation:


public IQueryable<SnowAlert> SomeOperation(string state)


   var events = (from p in this.CurrentDataSource.WeatherAlerts

        where p.State == state

                select p);

     return events;



Of course you will most likely want to intercept the request.  There are two interceptors. QueryInterceptor and ChangeInterceptor.  Let's look at both.  On my Reporting Service I had both interceptors.  While my implementation is hokey, it makes the point.  My QueryInterceptor will only return Evangelists with the name Clark Sell.  You could actually imagine a scenario where based on your security profile you filtered results.   My ChangeInterceptor fires for the Evangelists Entity and looks at the change operation.  If the Name being added is Clark Sell it throws an exception.



public Expression<Func<Evangelist, bool>> FilterEvangelists()


        return e => e.Name == "Clark Sell";




public void OnChangeEvangelists(Evangelist e, UpdateOperations operation)


        if (operation == UpdateOperations.Add '' operation == UpdateOperations.Change)


                if (e.Name == "Clark Sell")


                        throw new DataServiceException(400, "Sorry not allowed.");





So I mentioned earlier, you don't have to expose your Entity framework but your own object. That is true, but on your object you will have to expose IQueryable and IEditable. In my sample demo you can look at the WeatherAlertRepository.cs for a simple example.  Of course this is a place where your implementation will vary drastically.


Removing the SVC extension.

Extension in URLs are just so 2005.  Luckily it's really not a big deal to get rid of with ASP.NET. On App_Start in our global.asax we can actually register a Service Route.  This will add support for extension-less base addresses.

var factory = newDataServiceHostFactory();


newServiceRoute("WeatherAlerts", factory, typeof(WeatherAlerts)));

PHEWWWW, what a whirlwind. Here are some great resources to check out:


My CVCC Demo Source:

You can find my demo code at: https://bitbucket.org/csell5/demos/src/tip/cvcc/.  When I present I try to do three things. 

  1. Limit the use of slides.  Why?  Well PowerPoint just isn't what you live in.2. Keep it simple.  _Solutions that are too "complicated" could dilute concepts.3. _Make it more complicated than hello world.  Hello world is great in a lot of examples but the devil is always in the details.  I try to keep it real world. 

My demo was based on something I have been thinking about a lot, reporting.  Bottom line, this is a real world thing I am building.  There are four projects:

  1. DPE.Reporting.Web.Tests, its my test project2. DPE.Reporting.Web, all of the web assets.  This will include the OData endpoints3. DPE.Reporting.Infrastructure, glue in the middle4. DPE.Reporting.Core, core stuff including the models.




At this point my data model is pretty incomplete but it's enough.  In the demo we were dealing with the Evangelist Entity.




I also showed how you could just expose any type of object, not just an Entity Framework Model.  Since it snowed that day I created a Weather Alerting Service.  It just seemed appropriate. It was comprise of three things:

  1. SnowAlert.cs, found in Core and is the basic data model.2. WeatherAlertsRepository.cs, found in Infrastructure.  This is a very cheap repository like pattern to retrieve data. It's also the object that exposes the IQueryable for OData.3. WeatherAlerts.svc, found in the web project. This is the OData endpoint.


What's next?

Well for me, it's looking at the following:

  • Best practice's around JQuery integration* PUT and POST techniques for complicated data structures