Transforming JSON Using JSON.NET And JSONPath

I’ve been building REST APIs for the better part of the last three years. Those APIs have known representations and the responses are crafted to fulfill a plethora of scenarios to many different clients. For some clients, those responses may be too much information. The idea of “too much data” has made me ask:

What if the client could filter only the fields they need from the response representation?

I know what many are going to say?

What about GraphQL?

Yes, I am aware of GraphQL but the idea is to enhance existing REST APIs with minimal impact to clients.

The General Idea

REST APIs have a known state, which I like to call a representation. Ideally, this representation rarely changes once deployed, with enhancements adding new fields.

{
    'FirstName': 'John',
    'LastName': 'Smith',
    'Enabled': false,
    'Roles': [ 'User' ],
    'Results' : [ 1, 2, 3 ]
}

What if we defined a query in the HTTP Header that defined a new representation? And what if we could use JSONPath?

{
    'FirstName': '',
    'Role' : '$.Roles[0]'
}

The result we would get would look like:

{
    "FirstName": "John",
    "Role": "User"
}

Using JSON.NET

JSON.NET allows you to use JSONPath to select tokens. I have written a quick proof of concept that takes a template and selects the fields.

using System;
using Newtonsoft.Json.Linq;

namespace ConsoleApplication
{
    class Program
    {
        static void Main(string[] args)
        {
            var original = JObject.Parse(@"{
               'FirstName': 'John',
               'LastName': 'Smith',
               'Enabled': false,
               'Roles': [ 'User' ],
               'Results' : [ 1, 2, 3]
             }");
            var template = JObject.Parse(@"{
              'FirstName': '',
              'Role' : '$.Roles[0]'
            }");

            var result = Process(original, template);

            Console.WriteLine(result.ToString());
            /*
              {
                "FirstName": "John",
                "Role": "User"
              }
             */
        }

        private static JObject Process(JObject original, JObject template)
        {
            var result = new JObject();

            foreach (var property in template.Properties())
            {
                var value = property.Value?.ToString();

                var path = value?.StartsWith("$") == true
                    ? value
                    : property.Path;

                var selected = original.SelectToken(path);
                result.Add(property.Name, selected);
            }

            return result;
        }
    }
}

How Would We Use This?

I was thinking about writing a piece of OWIN or ASP.NET Core Middleware that can intercept any JSON response and transform it if an X-JSON-TEMPLATE header is present in the request. There are advantages to this approach:

1. Can be layered on top of an existing REST API
2. Implementation agnostic
3. Reduce network payload

There are also cons to this idea:

1. The backend still needs to build the entire response
2. Different transformations mean different responses, which breaks web caching
3. Incorrect transformations may return incorrect results
4. Size limits of HTTP Headers are normally 8kb-16kb

Conclusion

While there may be similarities with GraphQL, this approach doesn’t throw out any existing REST APIs but allows them to be enhanced with similar functionality. I still need to sit down and write the middleware, and I’m excited to do so soon. Stay tuned, but if you beat me to it please leave a link so I can…. borrow it :)