*RWYU* - Request What You Use?

One thing that always bothers me is useless or expensive RPCs, and the various ad-hoc ways of filtering proto fields.  So I was  inspired by IWYU and came up with these ideas...

** When making an RPC include a 'template' for the response.  The simplest possible thing is an actual response object.  Fields you want have defaultValue, fields you don't want are empty.  Call it the 'Mask' or 'Filter'

    FooResponse = fooRpc(FooRequest)

becomes

   FooResponse = fooRpc(FooRequest request, FooResponse mask)

** Building up these masks can be tricky.  So borrow code from dremel so you can make them look like a SELECT.

   mask = MaskMaker.create(fooResponse, "message.*,foo.name");

** A Rpc service can then look at the fields wanted and adjust it's behavior.  If a proto message isn't needed you can probably save extra downstream RPCs.  The logic is actually fairly clean:

   if (mask.hasMessage()) {
     if (mask.getMessage().hasField()) {
       m.setField(x);
     }
     response.setMessage(m);
   }

 [and you could conceive of DSLs or other mechanisms that dispatch producer logic for portions of the response.]

** A client can enforce the filter/mask by stripping out anything the server sends that it did not expect.  Tracking mismatches here can identify efficiency wins.

** To make creating masks easier we could sample RPCs and find a maximal mask that includes data that is actually accessed. Just Instrument the getters on the generated proto objects and figure out what your code actually uses.  (Obvious caveats apply about exceptional cases, speculative RPCs, etc.)

Anyone tried to solve this problem in a general way?  Am I crazy to think that there's a general-purpose solution to this problem?

[Bonus points if we can figure out a way to also specify limits and continuation tokens inside a mask.  Maybe with proto extensions?]




#googplus

#googplus