Per the Design reference, a B-Tree index associated with a Text type field stores the first 1024 characters of the field (maximum). If an indexed string contains more than 1024 characters, then searches for strings beyond the 1024th character will fail.
For most text data, storing only the first 1024 characters is sufficient as text fields to be indexed, in general, contain small amounts of text, meaning only a few words. Larger text fields, meaning fields that contain many words, are typically better suited to have Keyword indexes.
However, there may be instances where a text field will have 1024+ characters and a Keyword index is not a good fit. Say for example, the searches will need to contain characters that are considered word boundaries (such as space).
In these cases, rather than performing a sequential search, there are other options:
- Use a keyword index with the following workflow:
- Store the search string in a text field
- Run the command DISTINCT VALUES on the text field
- Run a query using each item returned from DISTINCT VALUES
- Using a keyword index, break up the search string into its individual words and perform a query using each word. This should significantly reduce the selection of possible results. Then run a query within the selection for the specific search string.
- Instead of storing all the text in a single text field, distribute the text to a series of records in a related table. In all likelihood, this would lead to text fields less than 1024 characters, thus enabling the use of B-Tree indexes. Of course, this would only be applicable in certain cases, where the text was in a certain structured format already.
(This technique is discussed in depth in the Tech Note Multi-Keyword Search)