Fully Support Sparse Fieldsets

Whilst the `QueryParser` was correctly identifying requested fieldsets,
nothing was done to actually support this.

This change prunes returned fields to those requested should it be the
case.

Note that this change also includes a few more typespecs for functions I
touched or read.

Resolves #120
Closes #156

README.md

@@ -56,9 +56,12 @@ defmodule MyApp.PostView do
 end
 ```
 
-is an example of a basic view. You can now call `render(conn, MyApp.PostView, "show.json", %{data: my_data, meta: meta})` or `'index.json` normally.
+is an example of a basic view. You can now call
+`render(conn, MyApp.PostView, "show.json", %{data: my_data, meta: meta})`
+or `"index.json"` normally.
 
-If you'd like to use this without phoenix simply use the `JSONAPI.View` and call `JSONAPI.Serializer.serialize(MyApp.PostView, data, conn, meta)`.
+If you'd like to use this without Phoenix simply use the `JSONAPI.View` and call
+`JSONAPI.Serializer.serialize(MyApp.PostView, data, conn, meta)`.
 
 ## Parsing and validating a JSONAPI Request
 
@@ -71,18 +74,19 @@ plug JSONAPI.QueryParser,
   view: PostView
 ```
 
-This will add a `JSONAPI.Config` struct called `jsonapi_config` to your conn.assigns. If a user tries to
-sort, filter, include, or sparse fieldset an invalid field it will raise a plug error that shows the
-proper error message.
+This will add a `JSONAPI.Config` struct called `jsonapi_config` to your
+`conn.assigns`. If a user tries to sort, filter, include, or requests an
+invalid fieldset it will raise a `Plug` error that shows the proper error
+message.
 
-The config holds the values parsed into things that are easy to pass into an Ecto query, for example
+The config holds the values parsed into things that are easy to pass into an Ecto
+query, for example `sort=-name` will be parsed into `sort: [desc: :name]` which
+can be passed directly to the `order_by` in Ecto.
 
-`sort=-name` will be parsed into `sort: [desc: :name]` which can be passed directly to the order_by in ecto.
+This sort of behavior is consistent for includes.
 
-This sort of behavior is consistent for includes. Sparse fieldsets happen in the view using Map.take but
-when Ecto gets more complex field selection support we will go further to only query the data we need.
-
-You will need to handle filtering yourself, the filter is just a map with key=value.
+The `JSONAPI.QueryParser` plug also supports [sparse fieldsets](https://jsonapi.org/format/#fetching-sparse-fieldsets).
+Please see its documentation for details.
 
 ## Camelized or Dasherized Fields
 

lib/jsonapi/plugs/query_parser.ex

@@ -4,22 +4,24 @@ defmodule JSONAPI.QueryParser do
   alias JSONAPI.Exceptions.InvalidQuery
   alias Plug.Conn
   import JSONAPI.Utils.IncludeTree
+  import JSONAPI.Utils.String, only: [underscore: 1]
 
   @moduledoc """
-  Implements a fully JSONAPI V1 spec for parsing a complex query string and returning elixir
-  datastructures. The purpose is to validate and encode incoming queries and fail quickly.
+  Implements a fully JSONAPI V1 spec for parsing a complex query string and
+  returning Elixir datastructures. The purpose is to validate and encode incoming
+  queries and fail quickly.
 
   Primarialy this handles:
     * [sorts](http://jsonapi.org/format/#fetching-sorting)
     * [include](http://jsonapi.org/format/#fetching-includes)
     * [filtering](http://jsonapi.org/format/#fetching-filtering)
-    * [sparse fieldsets](http://jsonapi.org/format/#fetching-includes)
+    * [sparse fieldsets](https://jsonapi.org/format/#fetching-sparse-fieldsets)
     * [pagination](http://jsonapi.org/format/#fetching-pagination)
 
-  This plug works in conjunction with a JSONAPI View as well as some plug defined
-  configuration.
+  This Plug works in conjunction with a `JSONAPI.View` as well as some Plug
+  defined configuration.
 
-  In your controller you may add
+  In your controller you may add:
 
   ```
   plug JSONAPI.QueryParser,
@@ -29,10 +31,13 @@ defmodule JSONAPI.QueryParser do
   ```
 
   If your controller's index function receives a query with params inside those
-  bounds it will build a JSONAPI.Config that has all the validated and parsed
-  fields for your usage. The final configuration will be added to assigns `jsonapi_query`.
+  bounds it will build a `JSONAPI.Config` that has all the validated and parsed
+  fields for your usage. The final configuration will be added to assigns
+  `jsonapi_query`.
+
+  The final output will be a `JSONAPI.Config` struct and will look similar to the
+  following:
 
-  The final output will be a `JSONAPI.Config` struct and will look similar to like
       %JSONAPI.Config{
         view: MyView,
         opts: [view: MyView, sort: ["created_at", "title"], filter: ["title"]],
@@ -50,8 +55,13 @@ defmodule JSONAPI.QueryParser do
       }
 
   The final result should allow you to build a query quickly and with little overhead.
-  You will notice the fields section is a not as easy to work with as the others and
-  that is a result of Ecto not supporting high quality selects quite yet. This is a WIP.
+
+  ## Spare Fieldsets
+
+  Sparse fieldsets are supported. By default your response will include all
+  available fields. Note that the query to your database is left to you. Should
+  you want to query your DB for specific fields `JSONAPI.Config.fields` will
+  return the requested fields for each resource (see above example).
 
   ## Options
     * `:view` - The JSONAPI View which is the basis for this plug.
@@ -67,10 +77,12 @@ defmodule JSONAPI.QueryParser do
   For more details please see `JSONAPI.UnderscoreParameters`.
   """
 
+  @impl Plug
   def init(opts) do
     build_config(opts)
   end
 
+  @impl Plug
   def call(conn, opts) do
     query_params_config_struct =
       conn
@@ -94,6 +106,7 @@ defmodule JSONAPI.QueryParser do
   def parse_pagination(%Config{} = config, page),
     do: Map.put(config, :page, struct_from_map(page, %Page{}))
 
+  @spec parse_filter(Config.t(), keyword()) :: Config.t()
   def parse_filter(config, map) when map_size(map) == 0, do: config
 
   def parse_filter(%Config{opts: opts} = config, filter) do
@@ -111,7 +124,8 @@ defmodule JSONAPI.QueryParser do
     end
   end
 
-  def parse_fields(config, map) when map_size(map) == 0, do: config
+  @spec parse_fields(Config.t(), map()) :: Config.t() | no_return()
+  def parse_fields(%Config{} = config, fields) when fields == %{}, do: config
 
   def parse_fields(%Config{} = config, fields) do
     Enum.reduce(fields, config, fn {type, value}, acc ->
@@ -121,9 +135,14 @@ defmodule JSONAPI.QueryParser do
         |> Enum.into(MapSet.new())
 
       requested_fields =
-        value
-        |> String.split(",")
-        |> Enum.into(MapSet.new(), &String.to_atom/1)
+        try do
+          value
+          |> String.split(",")
+          |> Enum.map(&underscore/1)
+          |> Enum.into(MapSet.new(), &String.to_existing_atom/1)
+        rescue
+          ArgumentError -> raise_invalid_field_names(value, config.view.type())
+        end
 
       unless MapSet.subset?(requested_fields, valid_fields) do
         bad_fields =
@@ -132,10 +151,7 @@ defmodule JSONAPI.QueryParser do
           |> MapSet.to_list()
           |> Enum.join(",")
 
-        raise InvalidQuery,
-          resource: config.view.type(),
-          param: bad_fields,
-          param_type: :fields
+        raise_invalid_field_names(bad_fields, config.view.type())
       end
 
       %{acc | fields: Map.put(acc.fields, type, MapSet.to_list(requested_fields))}
@@ -220,18 +236,20 @@ defmodule JSONAPI.QueryParser do
     end
   end
 
-  def get_valid_fields_for_type(%{view: view}, type) do
+  @spec get_valid_fields_for_type(Config.t(), String.t()) :: list(atom())
+  def get_valid_fields_for_type(%Config{view: view}, type) do
     if type == view.type do
       view.fields
     else
       get_view_for_type(view, type).fields
     end
   end
 
+  @spec get_view_for_type(module(), String.t()) :: module() | no_return()
   def get_view_for_type(view, type) do
     case Enum.find(view.relationships, fn {k, _v} -> Atom.to_string(k) == type end) do
       {_, view} -> view
-      nil -> raise InvalidQuery, resource: view.type, param: type, param_type: :fields
+      nil -> raise_invalid_field_names(type, view.type())
     end
   end
 
@@ -241,6 +259,12 @@ defmodule JSONAPI.QueryParser do
     raise InvalidQuery, resource: resource_type, param: param, param_type: :include
   end
 
+  @spec raise_invalid_field_names(bad_fields :: String.t(), resource_type :: String.t()) ::
+          no_return()
+  defp raise_invalid_field_names(bad_fields, resource_type) do
+    raise InvalidQuery, resource: resource_type, param: bad_fields, param_type: :fields
+  end
+
   defp build_config(opts) do
     view = Keyword.fetch!(opts, :view)
     struct(Config, opts: opts, view: view)

lib/jsonapi/view.ex

@@ -56,6 +56,23 @@ defmodule JSONAPI.View do
         def relationships, do: []
       end
 
+  Fields may be omitted manually using the `hidden/1` function.
+
+      defmodule UserView do
+        use JSONAPI.View
+
+        def fields, do: [:id, :username, :email]
+
+        def type, do: "user"
+
+        def hidden(_data) do
+          [:email] # will be removed from the response
+        end
+      end
+
+  In order to use [sparse fieldsets](https://jsonapi.org/format/#fetching-sparse-fieldsets)
+  you must include the `JSONAPI.QueryParser` plug.
+
   ## Relationships
 
   Currently the relationships callback expects that a map is returned
@@ -94,6 +111,9 @@ defmodule JSONAPI.View do
   The default behaviour for `host` and `scheme` is to derive it from the `conn` provided, while the
   default style for presentation in names is to be underscored and not dashed.
   """
+
+  alias Plug.Conn
+
   defmacro __using__(opts \\ []) do
     {type, opts} = Keyword.pop(opts, :type)
     {namespace, _opts} = Keyword.pop(opts, :namespace)
@@ -120,10 +140,37 @@ defmodule JSONAPI.View do
         def namespace, do: Application.get_env(:jsonapi, :namespace, "")
       end
 
-      def attributes(data, conn) do
-        hidden = hidden(data)
+      defp requested_fields_for_type(%Conn{assigns: %{jsonapi_query: %{fields: fields}}} = conn) do
+        fields[type()]
+      end
+
+      defp requested_fields_for_type(_conn), do: nil
+
+      defp fields_for_type(requested_fields, fields) when requested_fields in [nil, %{}],
+        do: fields
+
+      defp fields_for_type(requested_fields, fields) do
+        fields
+        |> MapSet.new()
+        |> MapSet.intersection(MapSet.new(requested_fields))
+        |> MapSet.to_list()
+      end
+
+      @spec visible_fields(map(), conn :: nil | Conn.t()) :: list(atom)
+      def visible_fields(data, conn) do
+        all_fields =
+          conn
+          |> requested_fields_for_type()
+          |> fields_for_type(fields())
+
+        hidden_fields = hidden(data)
 
-        visible_fields = fields() -- hidden
+        all_fields -- hidden_fields
+      end
+
+      @spec attributes(map(), conn :: nil | Conn.t()) :: map()
+      def attributes(data, conn) do
+        visible_fields = visible_fields(data, conn)
 
         Enum.reduce(visible_fields, %{}, fn field, intermediate_map ->
           value =

test/jsonapi/plugs/query_parser_test.exs

@@ -119,6 +119,10 @@ defmodule JSONAPI.QueryParserTest do
     assert_raise InvalidQuery, "invalid fields, blag for type mytype", fn ->
       parse_fields(config, %{"mytype" => "blag"})
     end
+
+    assert_raise InvalidQuery, "invalid fields, username for type mytype", fn ->
+      parse_fields(config, %{"mytype" => "username"})
+    end
   end
 
   test "get_view_for_type/2" do

test/jsonapi/view_test.exs

@@ -189,7 +189,28 @@ defmodule JSONAPI.ViewTest do
     assert data.meta.total_pages == 100
   end
 
-  test "attributes/2 does not display hidden fields with deprecated hidden/0" do
+  test "visible_fields/2 returns all field names by default" do
+    data = %{age: 100, first_name: "Jason", last_name: "S", password: "securepw"}
+
+    assert [:age, :first_name, :last_name, :full_name] ==
+             UserView.visible_fields(data, %Plug.Conn{})
+  end
+
+  test "visible_fields/2 removes any hidden field names" do
+    data = %{title: "Hidden body", body: "Something"}
+
+    assert [:title] == PostView.visible_fields(data, %Plug.Conn{})
+  end
+
+  test "visible_fields/2 trims returned field names to only those requested" do
+    data = %{body: "Chunky", title: "Bacon"}
+    config = %JSONAPI.Config{fields: %{PostView.type() => [:body]}}
+    conn = %Plug.Conn{assigns: %{jsonapi_query: config}}
+
+    assert [:body] == PostView.visible_fields(data, conn)
+  end
+
+  test "attributes/2 does not display hidden fields" do
     expected_map = %{age: 100, first_name: "Jason", last_name: "S", full_name: "Jason S"}
 
     assert expected_map ==
@@ -215,4 +236,12 @@ defmodule JSONAPI.ViewTest do
                nil
              )
   end
+
+  test "attributes/2 can return only requested fields" do
+    data = %{body: "Chunky", title: "Bacon"}
+    config = %JSONAPI.Config{fields: %{PostView.type() => [:body]}}
+    conn = %Plug.Conn{assigns: %{jsonapi_query: config}}
+
+    assert %{body: "Chunky"} == PostView.attributes(data, conn)
+  end
 end