This document contains various suggestions and solutions for dealing with "Hierarchical Facets" - a concept which can mean differnet things to differnet people depending on their data.
Transcribed (with slight edits) from Mastering the Power of Faceted Search by Chris Hostetter of Lucid Imagination (starting ~28min in)
For simple categories, facet.field works fine as-is. However, categorization schemes are frequently organized in a hierarchically-structured scheme, and the user experience for interacting with that taxonomy involves drill down, starting at the top (more general) and whittling the way down (more specific).
This is a basic approach that works well for most use cases and takes advantage of basic Solr faceting parameters by encoding the facet terms at index time.
Doc#1: NonFic > Law Doc#2: NonFic > Sci Doc#3: NonFic > Hist, NonFic > Sci > Phys |
In this example, we have documents associated with multiple categories, like Doc#3. We also have documents that are mapped to internal nodes, like Doc#2.
You must perform some index time processing on this flattened data in order to create the tokens needed for a facet.prefix approach. When we index the data we create specially formatted terms that encode the depth information for each node that appears as part of the path, and include the hierarchy separated by a common separator (“depth/first level term/second level term/etc”). We also add additional terms for every ancestors in the original data.
Doc#1: 0/NonFic, 1/NonFic/Law Doc#2: 0/NonFic, 1/NonFic/Sci Doc#3: 0/NonFic, 1/NonFic/Hist, 0/NonFic, 1/NonFic/Sci, 2/NonFic/Sci/Phys |
In case you are indexing terms that may have another term in the beginning, adding a separator at the end of each term helps distinquish these terms:
Doc#1: 0/Books/, 1/Books/Book/ Doc#2: 0/Books/, 1/Books/BookPart/ |
Then in the query always include a trailing slash, e.g. "facet.prefix = 1/Books/Book/" to avoid matching "1/Books/BookParts".
With this type of index data, we can then go on and query this to get a drill-down. Initially, we can say we want to facet on the category field with the facet.prefix “1/NonFic”: things that are children of NonFic at a depth of 1.
facet.field = category facet.prefix = 1/NonFic facet.mincount = 1 |
<result numFound=”3” ... <lst name=”facet_fields”> <lst name=”category”> <int name=”1/NonFic/Sci”>2</int> <int name=”1/NonFic/Hist”>1</int> <int name=”1/NonFic/Law”>1</int> |
If we drill down into NonFic/Sci, we just add the fq (filter query) as normal and tweak the facet.prefix from the children 1/NonFic to the children of 2/NonFic/Sci
fq = {!raw f=category}1/NonFic/Sci facet.field = category facet.prefix = 2/NonFic/Sci facet.mincount = 1 |
<result numFound=”2” ... <lst name=”facet_fields”> <lst name=”category”> <int name=”2/NonFic/Sci/Phys”>1</int> |
We’ve used the depth prefix that lets us look one level deep, but by tweaking the encoding, alternative user experiences can be created.
The solr.PathHierarchyTokenizerFactory is designed to output file path hierarchies as synonyms, but can also be used in other simple hierarchies.
Doc #1: /usr/local/apache Doc #2: /etc/apache2 Doc #3: /etc/apache2/conf.d |
Doc #1: /usr, /usr/local, /usr/local/apache Doc #2: /etc, /etc/apache2 Doc #3: /etc, /etc/apache2, /etc/apache2/conf.d |
facet.field = category facet.mincount = 1 |
<result numFound=”3” ... <lst name=”facet_fields”> <lst name=”category”> <int name=”/etc”>2</int> <int name=”/etc/apache2”>2</int> <int name=”/etc/apache2/conf.d”>1</int> <int name=”/usr”>1</int> <int name=”/usr/local”>1</int> <int name=”/usr/local/apache”>1</int> |
Unlike the facet.prefix approach, it isn’t as easy to constrain the depth of the taxonomies, but for small numbers of terms this may be a good approach.
SOLR-792] [https://issues.apache.org/jira/browse/SOLR-792
Pivot facets are query time constructs that allow arbitrary facet results, but they should be used wisely to avoid performance bottlenecks.
You can think of it as "Decision Tree Faceting" which tells you in advance what the "next" set of facet results would be for a field if you apply a constraint from the current facet results, e.g. "for facet A, the constraints/counts are X/N, Y/M,” and if you were to constrain A by X, then the constraint counts for B would be S/P, T/Q, etc. Another way to think of it is each field is treated as a vector containing the constraint counts for that field, and taking a "cross product" to produce an N-dimensional matrix showing the counts for each permutation.
This feature can be easily applied to hierarchical facets in some cases, particularly those where a particular document only appears at one point in the taxonomy.
Doc#1: NonFic > Law Doc#2: NonFic > Sci Doc#3: NonFic > Sci > Phys |
At index time, we split the data into a separate field for each level of the hierarchy.
Doc#1: category_level0: NonFic; category_level1: Law Doc#2: category_level0: NonFic; category_level1: Sci Doc#3: category_level0: NonFic; category_level1: Sci, category_level2:Phys |
When querying Solr, we specify the facet.pivot parameter, which is a comma-separated list of fields to “pivot” on:
facet.pivot = category_level0,category_level1,category_level2 |
<result numFound=”3” ... <lst name=”facet_pivot”> <arr name=”category_level0,category_level1,category_level2”> <lst> <str name=”field”>category_level0</str> <str name=”value”>NonFic</str> <int name=”count”>3</int> <arr name=”pivot”> <lst> <str name=”field”>category_level1</str> <str name=”value”>Law</str> <int name=”count”>1</int> </lst> <lst> <str name=”field”>category_level1</str> <str name=”value”>Sci</str> <int name=”count”>2</int> <arr name=”pivot”> <lst> <str name=”field”>category_level2</str> <str name=”value”>Phys</str> <int name=”count”>1</int> </lst> </arr> </lst> </arr> |
Strict Facet Hierarchies:
Hierarchical faceting with slow startup, low memory overhead and fast response. Distinguishing features as compared to SOLR-64 and SOLR-792 are
This is a shell around LUCENE-2369, making it work with the Solr API. The underlying principle is to reference terms by their ordinals and create an index wide documents to tags map, augmented with a compressed representation of hierarchical levels.
https://issues.apache.org/jira/browse/LUCENE-3079
TBD (To Be Documented)