Adding OData Inline Count Support to the ASP.NET Web API

Note: This post assumes that you’re working against the NuGet bits as of 4/5/2012, in which OData support is limited to $top, $skip, $orderby, and $filter. I plan to go back and check whether the latest bits already have $inlinecount support now that the ASP.NET Web Stack has been open-sourced.

Update: Moving older updates to the bottom of the post. Short version: this worked in theory, but absolutely has quirks as it is admittedly an experimental hack. It explodes if you use EF (initial testing did not). Use with caution. There’s also a comment from Marcin Dobosz if you’re interested in what the Web API team is looking at for OData support.

Surely this is fair use. Oh please let this be fair use.

As the note above implies, the ASP.NET Web API does not (or did not) support the $inlinecount OData parameter. Many have pointed out that this defeats much of the client-side paging advantages of OData. Indeed, a friend of mine has personally run up against this. He wants to show the user how many total pages are possible; he can say that page six should be “skip 50 and take 10” all day long, but without the total count, he cannot calculate the maximum page number.

To demonstrate the problem, take a look at this Fiddler request. We ask for the total count of items, but the default ASP.NET Web API bits do not deliver the information!

The total count is not provided

When we’re done, we will have the count!

We have inserted the count!

Even if we apply a filter and a skip / take, we will know how many results we should plan to receive if we page through everything. Now we can divide that by the page size and know exactly how many pages the user can choose from.

[Aside: The merits of pagination are not the topic of this post. While I tend to agree with Jeff Atwood that paradigms are shifting, many still prefer - or are bound by their customers - to provide a page list].

One way around this was to simply make a second call to the server, parse out the $filter, and call Count(). This is not very elegant. While what I present below might not be the best possible approach, it does get us back down to a single call to the server which returns the $inlinecount as requested.

Let’s get to the code.

I chose to implement this as a message handler, so the first and simplest thing to do is register a new InlineCountHandler. I was working in a self-hosted Web API project, so I did this while setting up the configuration:


config.MessageHandlers.Add(new InlineCountHandler());

I had originally tried returning an anonymous type like this


new { Count = unpagedResults.Count(), Results = pagedResults }
// also tried pagedResults.ToArray() and a variety of other things

but this kept resulting in a 504. I decided to just go ahead and make a type to wrap our result. Trial and error also showed that our Result property needed to be an array (i.e., do not pass along the IQueryable) and cannot be object[] (i.e., we need to specify the proper type).


public class ResultValue<T>
{
    public int Count { get; set; }
    public T[] Results { get; set; }
}

After that it was just a matter of capturing the base response and adding the count in an override of SendAsync. If $inlinecount isn’t present and set to “allpages”, we just forward along the base response.


private bool ShouldInlineCount(HttpRequestMessage request)
{
    var queryParams = request.RequestUri.ParseQueryString();
    var inlinecount = queryParams["$inlinecount"];
    return string.Compare(inlinecount, "allpages", true) == 0;
}

Otherwise, we press on by adding a continuation to the base task. We are only going to bother working our magic if the response was a 200 OK and resulted in an ObjectContent of IQueryable<>.


private bool ResponseIsValid(HttpResponseMessage response)
{
    // Only do work if the response is OK
    if (response == null || response.StatusCode != HttpStatusCode.OK) return false;

    // Only do work if we are an ObjectContent
    return response.Content is ObjectContent;
}

Type queriedType;
// Can we find the underlying type of the results?
if (pagedResultsValue is IQueryable)
    queriedType = ((IQueryable)pagedResultsValue).ElementType;
else
    return response;

I haven’t performance tested the following, but my theoretical understanding is we won’t be issuing any additional network traffic – this will just resend the modified request inside our own pipeline. Additionally, the additional database call should be limited to a SELECT COUNT, so not too much additional chatter. At any rate, it works, which is step one.


// Reissue the request without a skip/take to get our count. This will preserve filtering which
// could affect the count
var newRequest = new HttpRequestMessage(
 request.Method,
 request.RequestUri.AbsoluteUri.Replace("$skip=", "$_skip=").Replace("$top=", "$_top="));

// Get the result with no paging
var unpagedTaskResult = base.SendAsync(newRequest, cancellationToken).Result;
var unpagedResultsValue = this.GetValueFromObjectContent(unpagedTaskResult.Content);

We don’t know what custom types our controllers might end up delivering as the T in IQueryable<T>, so we get to play with some generics and dynamic invocation.


var resultsValueMethod =
 this.GetType().GetMethod("CreateResultValue", BindingFlags.Instance | BindingFlags.NonPublic).MakeGenericMethod(new[] { queriedType });
 // Create the result value with dynamic type
var resultValue = resultsValueMethod.Invoke(
 this, new[] { unpagedResultsValue, pagedResultsValue });

Finally, we reset the Content to our ResultValue.


// Push the new content and return the response
response.Content = CreateObjectContent(resultValue, response.Content.Headers.ContentType);
return response;

So that’s what I came up with. You can get the full source code on GitHub. You’ll want the feature/inlinecount branch.

I’d be more than happy for someone to tear this apart and provide a better solution. Until then, it is what it is. I’ll almost certainly cover this at my upcoming ASP.NET Web API talk at HUNTUG in Huntsville, AL on 4/10/2012. Come out and see me!

Update: For whatever reason, this breaks content negotiation to XML. If you issue requests without $inlinecount with an Accept header for XML, it’s all groovy. Issue it with $inlinecount so that we intercept and add the Count, it still comes out as JSON. Be aware.

Update: XML Content Negotiation is fixed now. Silly me, forgot to use the ctor that preserves the MediaTypeHeaderValue. Other refactorings are also present in the latest code on GitHub, some of which has been incorporated into this post. Also note Marcin’s comment about the approach the Web API team is taking (still not slated for V1).

Another update: As it currently stands, this only works in a self hosted environment. If you host your Web API in an MVC website, for example, the base.SendAsync(newRequest… results in a 404 and bad things happen due to lack of error handling (hey, I told you this was not very elegant!). Anyway, Chris tells me he can get past the 404 by copying more than just the method and Uri to the request; however, his IOC breaks after that. You’ve been warned. ;) This has been fixed on the develop branch, although the general warning that this isn’t elegant stands. Proof of concept, experiment, hack; any of these terms are very reasonable.

And another: As Dante found, the “this has been fixed” part of the above update did eliminate  the exception when web hosting, but it also destroyed the accuracy. I have reverted the source on GitHub, and this will again only work in a self hosted environment.

About these ads

About David Ruttka

I've been "making computers do things" since I first saw King's Quest on a 286 PC in the mid-80's, but I turned it into a career just over a decade ago. While the majority of my experience has been on the Microsoft stack (C#, .NET, ASP.NET), I've recently been diving deeper into JavaScript and exploring the Ruby universe. Occasionally, I'll do a public speaking gig or write a blog post. When I'm not coding, I enjoy spending time with my family, watching hockey, and playing the occasional video game. You can also find me on Stack Overflow, Google Plus, and Twitter. Microsoft Certified Programming in HTML5 with JavaScript and CSS3 Specialist; MCPD Windows Developer 3.5
This entry was posted in Development and tagged , , , , . Bookmark the permalink.

10 Responses to Adding OData Inline Count Support to the ASP.NET Web API

  1. marcindobosz says:

    Hi David,
    Great post. I’m one of the developers on Web API and I thought you might like to know that we have some prototype code to support inline counts here: http://aspnetwebstack.codeplex.com/SourceControl/changeset/view/88372a0b4ab9#src%2fMicrosoft.Web.Http.Data%2fQueryFilterAttribute.cs.
    It’s implemented as a combination of an action filter that performs the count and a special ApiController base type that is responsible for building a result object that includes the count and the results.
    The feature is not currently scheduled to ship for V1 as part of the core Web API but I hope the code provides some inspiration.

  2. nimble99 says:

    Thanks David. I have built on your code to provide inlinecount functionality on top of RavenDB – and it works well: http://stackoverflow.com/questions/10544377/returning-iqueryable-but-need-raven-stats-to-insert-totalresults-header/10575932#10575932

  3. Dante Profeta says:

    Thank you David. Great post.
    I downloaded your sources from Git and performed basic tests. I found an odd behaviour:
    http://localhost:8081/api/Speaker?$inlinecount=allpages&$skip=2&$top=1
    returns
    {“Count”:1,”Results”:[{"Fame":100,"Id":3,"Name":"Dale Cooper"}]}

    Shouldn’t “Count” must be equals to 4 ?

    Haven’t debugged the code, but it seems reissuing the request with hacked “_skip” and “_top” doesn’t do the job.

    • David Ruttka says:

      Well, that’s embarrassing =) I really thought I tested such a case, but either I did not, or I wasn’t paying attention, or I broke something later.

      I did make some changes after writing this post, so I will try to look at the history and see if this did ever work. If so, when it fell apart.

      Thanks for letting me know about this.

    • David Ruttka says:

      Oooh.

      // HACK: We're going to temporarily update the RequestUri and put it back later. In previous
      // versions, we created a new HttpRequestMessage, but this saves us from a lot of cloning.
      // Use at your own risk ;)
      
    • David Ruttka says:

      Ok. Sure enough, I got this working when self-hosted by reverting to the old way.

      var newRequest = new HttpRequestMessage(
          request.Method,
          request.RequestUri.AbsoluteUri.Replace("$skip=", "$_skip=").Replace("$top=", "$_top="));
      
      var unpagedResult = base.SendAsync(newRequest, cancellationToken).Result;
      var unpagedResultsValue = this.GetValueFromObjectContent(unpagedResult.Content);
      

      http://localhost:8081/api/Speaker?$inlinecount=allpages&$skip=2&$top=1
      {“Count”:4,”Results”:[{"Fame":100,"Id":3,"Name":"Dale Cooper"}]}

      In hindsight, not re-using a request object makes a lot of sense to me. I’m not sure what ever possessed me to try it the other way!

      Update: When web-hosted, the second request comes back as a 404. That might be why I tried it the other way, and just never noticed that it broke the accuracy of the Count property. I have some other irons in the fire right now, so unfortunately that is as far as I’ll be able to take this today.

  4. Dante Profeta says:

    Hi David,
    Thank you to have rolled back your code.
    This post is a great resource indeed.

    • David Ruttka says:

      No problem. I’m going to push the update back to GitHub. I realize now that the WebHosted failure is indeed why I had changed it in the first place, but if accuracy is destroyed, then it was a bad change.

      I’m glad that at least some people are finding it useful, and thanks again for pointing out this error. Do also take a look at Marcin’s comment where the Web API team themselves are trying to tackle the inlinecount feature. Theirs might very well be more robust than mine! :)

Comments are closed.