To display a picture for contacts in Sitecore's Experience Profile, you must set the contact's
Avatar facet. However, beginning with Sitecore 9.1 Inital Release, that facet (or any facet with a
byte property) can completely break xConnect indexing for your Sitecore installation (and eventually will if you use it). Let's look at how to fix that.
If you are affected by this issue, the xConnect Indexer logs will be flooded with this error message:
[Error] Failed indexing next set of changes. There will be an attempt to recover from the failure.
System.FormatException: Invalid length for a Base-64 char array or string.
Deploy the following config patch to your xConnect Index Worker instance at
<?xml version="1.0" encoding="utf-8"?> <Settings> <Sitecore> <XConnect> <CollectionSearch> <Services> <IndexTruncationSettings> <Type>Sitecore.Xdb.Collection.Indexing.IndexTruncationSettings, Sitecore.Xdb.Collection</Type> <Options> <StringFieldMaximumLength>10920</StringFieldMaximumLength> </Options> </IndexTruncationSettings> </Services> </CollectionSearch> </XConnect> </Sitecore> </Settings>
⚠️Note⚠️: If you are patching Sitecore 9.1 (either release) or 9.2, set the
StringFieldMaximumLength value on line
32764 instead of
Restart the xConnect Index Worker job and indexing should resume.
Indexing the Avatar Facet
When you write an image to a contact's
Avatar facet, the property is stored in the data store (e.g., SQL Server) as JSON, and the
Picture property on the facet is stored as a base64-encoded string in that JSON.
Sometime after the contact and facet are written to the data store, xConnect's Index Worker attempts to index the facets for the contact into your indexing service (e.g., Solr). The Index Worker does this in the following steps:
- It reads the facet's JSON from the data store;
- It deserializes the JSON into the facet's model type found in the
App_Data\Modelsfolder of the Index Worker (
Sitecore.XConnect.Collection.Model, 9.X.jsonfor the
- It writes the facet to the index.
The problem is that when the Index Worker reads the JSON from the data store, it truncates the JSON properties down to the
StringFieldMaximumLength value found in the Index Worker's config at
App_Data\Config\Sitecore\CollectionSearch\sc.Xdb.Collection.Search.IndexTruncation.xml. When the Index Worker tries to model bind the base64-encoded string for the
Avatar facet, it will fail with the following exception if the
Picture property's length exceeds the
[Error] The attempt to recover from previous failure has not been successful. There will be another attempt. Attempts count: 135 System.FormatException: Invalid length for a Base-64 char array or string. at System.Convert.FromBase64_Decode(Char* startInputPtr, Int32 inputLength, Byte* startDestPtr, Int32 destLength) at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength) at System.Convert.FromBase64String(String s) at Sitecore.XConnect.Serialization.XObjectReader.ReadPrimitive(JsonReader reader, XdbPrimitiveTypeKind typeKind) at Sitecore.XConnect.Serialization.XObjectReader.ReadXObject(XdbType expectedType) at Sitecore.Xdb.Collection.Search.Solr.SolrResults.JsonParseExtensions.ToXObject(JObject jobject, XdbModel model, XdbType expectedType) at Sitecore.Xdb.Collection.Search.Solr.SolrResults.JsonParseExtensions.ToXObject(JObject jobject, XdbModel model, EntityType entityType, String facetKey) at Sitecore.Xdb.Collection.Search.Solr.DataRecordMapper.CreateFacetPropertyWithLastModified(KeyValuePair`2 facet) at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext() at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext() at System.Linq.Enumerable.<ConcatIterator>d__59`1.MoveNext() at Sitecore.Xdb.Collection.Search.Solr.DataRecordMapper.<ConvertToXObjects>d__30.MoveNext() at Sitecore.Xdb.Collection.Search.Solr.JObjectExpansion.JObjectPropertiesExpander.Expand(IEnumerable`1 properties) at Sitecore.Xdb.Collection.Search.Solr.DataRecordMapper.<MapObjects>b__13_0(DataRecord objectToIndex) at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext() at Sitecore.Xdb.Collection.Search.Solr.JsonPostCreator.CreateUpdateBatchJson(IEnumerable`1 docsToAdd) at Sitecore.Xdb.Collection.Search.Solr.JsonPostCreator.<>c__DisplayClass2_0.<CreateBatchUpdatePosts>b__0(IReadOnlyCollection`1 b) at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext() at Sitecore.Xdb.Collection.Search.Solr.SolrWriter.ParallelProcessing[T](IEnumerable`1 dataToProcess, Func`3 asyncFunc, SemaphoreSlim throttle, CancellationToken cancellationToken) at Sitecore.Xdb.Collection.Search.Solr.SolrWriter.SendJsonPostsToSolr(Uri updateUri, IEnumerable`1 jsonPosts, CancellationToken cancellationToken) at Sitecore.Xdb.Collection.Search.Solr.SolrWriter.<>c__DisplayClass14_0.<Write>b__0() at System.Threading.Tasks.Task`1.InnerInvoke() at System.Threading.Tasks.Task.Execute()
At this point, the Index Worker will continuously try and fail to process that facet, and it will not index any more records until the facet is either removed from the contact, or the
Picture property is shortened to a base64-encoded string that is less than the
StringFieldMaximumLength and divisible by
As a side note, this truncation behavior wasn't introduced until Sitecore 9.1 Initial Release, so prior versions of Sitecore were not affected.
Fixing the Issue
The reason the Index Worker is unable to model bind the
Avatar facet model is because base64-encoded strings must be evenly divisible by
4, and the
StringFieldMaximumLength values defined in Sitecore 9.1/9.2 (
32766) and 9.3 (
10922) are not divisible by
4 (these max-length values are related to Lucene's term-byte length limit). Thus the invalid length exception.
To prevent this exception, you can just change the
StringFieldMaximumLength value to the closest value that is divisible by
4 with a config patch:
32764 for Sitecore 9.1/9.2 and
10920 for Sitecore 9.3. With this change, your
Avatar facets will always be processed by the Index Worker, and any custom facets you create that might use a
byte property will be safely processed, too.
If you've looked closely at the
Avatar facet before, you may have noticed that the
Picture property is decorated with the
DoNotIndexAttribute which raises the question: why is the Index Worker processing this
Picture property at all? It's because the
DoNotIndexAttribute is not evaluated until after the
Avatar facet's JSON is bound to its model. So if the
Picture property is successfully deserialized, it won't actually be indexed.
There's an answer on Stack Exchange that suggests you can work around this issue by resizing the image below a certain size, but I've run into cases where resizing the images to the suggested size (and even smaller) would still cause the issue. Plus, resizing the
Avatar facet below 170x170 pixels makes the Experience Profile look bad 😀.
See the Fix in Action
I've got two repositories on GitHub that make use of the
Avatar facet, and both include the fix that I've described above. You can see them in action here:
- 🌋 Custom Timeline Eras: https://github.com/coreyasmith/sitecore-custom-timeline-eras
- 🎭 Experience Profile Gravatar: https://github.com/coreyasmith/sitecore-experience-profile-gravatar
In the Experience Profile Gravatar repository, I use Helix Publishing Pipeline (HPP) to automatically deploy the patch to the xConnect Index Worker on build, which you may find interesting if you're using HPP in your solution. See the
PublishSettings.XConnectIndexer.targets file and the configuration of the