To display a picture for contacts in Sitecore's Experience Profile, you must set the contact's Avatar facet. However, beginning with Sitecore 9.1 Inital Release, that facet (or any facet with a byte[] property) can completely break xConnect indexing for your Sitecore installation (and eventually will if you use it). Let's look at how to fix that.

TL;DR

If you are affected by this issue, the xConnect Indexer logs will be flooded with this error message:

[Error] Failed indexing next set of changes. There will be an attempt to recover from the failure.
System.FormatException: Invalid length for a Base-64 char array or string.

Deploy the following config patch to your xConnect Index Worker instance at \App_Data\jobs\continuous\IndexWorker\App_data\config\sitecore\z.IndexTruncationFix\sc.Xdb.Collection.Search.IndexTruncation.xml:

<?xml version="1.0" encoding="utf-8"?>
<Settings>
  <Sitecore>
    <XConnect>
      <CollectionSearch>
        <Services>
          <IndexTruncationSettings>
            <Type>Sitecore.Xdb.Collection.Indexing.IndexTruncationSettings, Sitecore.Xdb.Collection</Type>
            <Options>
              <StringFieldMaximumLength>10920</StringFieldMaximumLength>
            </Options>
          </IndexTruncationSettings>
        </Services>
      </CollectionSearch>
    </XConnect>
  </Sitecore>
</Settings>

⚠️Note⚠️: If you are patching Sitecore 9.1 (either release) or 9.2, set the StringFieldMaximumLength value on line 10 to 32764 instead of 10920.

Restart the xConnect Index Worker job and indexing should resume.

Indexing the Avatar Facet

When you write an image to a contact's Avatar facet, the property is stored in the data store (e.g., SQL Server) as JSON, and the byte[] Picture property on the facet is stored as a base64-encoded string in that JSON.

Sometime after the contact and facet are written to the data store, xConnect's Index Worker attempts to index the facets for the contact into your indexing service (e.g., Solr). The Index Worker does this in the following steps:

  1. It reads the facet's JSON from the data store;
  2. It deserializes the JSON into the facet's model type found in the App_Data\Models folder of the Index Worker (Sitecore.XConnect.Collection.Model, 9.X.json for the Avatar facet);
  3. It writes the facet to the index.

The problem is that when the Index Worker reads the JSON from the data store, it truncates the JSON properties down to the StringFieldMaximumLength value found in the Index Worker's config at App_Data\Config\Sitecore\CollectionSearch\sc.Xdb.Collection.Search.IndexTruncation.xml. When the Index Worker tries to model bind the base64-encoded string for the Avatar facet, it will fail with the following exception if the Picture property's length exceeds the StringFieldMaximumLength value:

[Error] The attempt to recover from previous failure has not been successful. There will be another attempt. Attempts count: 135
System.FormatException: Invalid length for a Base-64 char array or string.
   at System.Convert.FromBase64_Decode(Char* startInputPtr, Int32 inputLength, Byte* startDestPtr, Int32 destLength)
   at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength)
   at System.Convert.FromBase64String(String s)
   at Sitecore.XConnect.Serialization.XObjectReader.ReadPrimitive(JsonReader reader, XdbPrimitiveTypeKind typeKind)
   at Sitecore.XConnect.Serialization.XObjectReader.ReadXObject(XdbType expectedType)
   at Sitecore.Xdb.Collection.Search.Solr.SolrResults.JsonParseExtensions.ToXObject(JObject jobject, XdbModel model, XdbType expectedType)
   at Sitecore.Xdb.Collection.Search.Solr.SolrResults.JsonParseExtensions.ToXObject(JObject jobject, XdbModel model, EntityType entityType, String facetKey)
   at Sitecore.Xdb.Collection.Search.Solr.DataRecordMapper.CreateFacetPropertyWithLastModified(KeyValuePair`2 facet)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.<ConcatIterator>d__59`1.MoveNext()
   at Sitecore.Xdb.Collection.Search.Solr.DataRecordMapper.<ConvertToXObjects>d__30.MoveNext()
   at Sitecore.Xdb.Collection.Search.Solr.JObjectExpansion.JObjectPropertiesExpander.Expand(IEnumerable`1 properties)
   at Sitecore.Xdb.Collection.Search.Solr.DataRecordMapper.<MapObjects>b__13_0(DataRecord objectToIndex)
   at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
   at Sitecore.Xdb.Collection.Search.Solr.JsonPostCreator.CreateUpdateBatchJson(IEnumerable`1 docsToAdd)
   at Sitecore.Xdb.Collection.Search.Solr.JsonPostCreator.<>c__DisplayClass2_0.<CreateBatchUpdatePosts>b__0(IReadOnlyCollection`1 b)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Sitecore.Xdb.Collection.Search.Solr.SolrWriter.ParallelProcessing[T](IEnumerable`1 dataToProcess, Func`3 asyncFunc, SemaphoreSlim throttle, CancellationToken cancellationToken)
   at Sitecore.Xdb.Collection.Search.Solr.SolrWriter.SendJsonPostsToSolr(Uri updateUri, IEnumerable`1 jsonPosts, CancellationToken cancellationToken)
   at Sitecore.Xdb.Collection.Search.Solr.SolrWriter.<>c__DisplayClass14_0.<Write>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()

At this point, the Index Worker will continuously try and fail to process that facet, and it will not index any more records until the facet is either removed from the contact, or the Picture property is shortened to a base64-encoded string that is less than the StringFieldMaximumLength and divisible by 4.

As a side note, this truncation behavior wasn't introduced until Sitecore 9.1 Initial Release, so prior versions of Sitecore were not affected.

Fixing the Issue

The reason the Index Worker is unable to model bind the Avatar facet model is because base64-encoded strings must be evenly divisible by 4, and the StringFieldMaximumLength values defined in Sitecore 9.1/9.2 (32766) and 9.3 (10922) are not divisible by 4 (these max-length values are related to Lucene's term-byte length limit). Thus the invalid length exception.

To prevent this exception, you can just change the StringFieldMaximumLength value to the closest value that is divisible by 4 with a config patch: 32764 for Sitecore 9.1/9.2 and 10920 for Sitecore 9.3. With this change, your Avatar facets will always be processed by the Index Worker, and any custom facets you create that might use a byte[] property will be safely processed, too.

If you've looked closely at the Avatar facet before, you may have noticed that the Picture property is decorated with the DoNotIndexAttribute which raises the question: why is the Index Worker processing this Picture property at all? It's because the DoNotIndexAttribute is not evaluated until after the Avatar facet's JSON is bound to its model. So if the Picture property is successfully deserialized, it won't actually be indexed.

There's an answer on Stack Exchange that suggests you can work around this issue by resizing the image below a certain size, but I've run into cases where resizing the images to the suggested size (and even smaller) would still cause the issue. Plus, resizing the Avatar facet below 170x170 pixels makes the Experience Profile look bad 😀.

See the Fix in Action

I've got two repositories on GitHub that make use of the Avatar facet, and both include the fix that I've described above. You can see them in action here:

In the Experience Profile Gravatar repository, I use Helix Publishing Pipeline (HPP) to automatically deploy the patch to the xConnect Index Worker on build, which you may find interesting if you're using HPP in your solution. See the PublishSettings.XConnectIndexer.targets file and the configuration of the CoreyAndRick.Project.Common.XConnectIndexer project.