LEVIATHAN SYSTEMS

Buyer's Guide

A Valid Text Prefix Can Still Split a Grapheme Cluster

Technical contentPublished Updated

A valid UTF-8 prefix can stop inside a default extended grapheme cluster. Keeping complete code points preserves their encodings; keeping complete original clusters requires a different stopping boundary. For an AI text limit, agree the counted unit, the segmentation version and the permitted response when requirements conflict. In TXT-57, the supplied text contains 12 bytes, five Unicode scalar values and three default clusters. Its three-code-point prefix is valid UTF-8 but splits the original regional-indicator pair.

Specify both the unit and the permitted stopping boundary

A text field described as limited to a number of characters leaves a purchasing question unresolved. Does the supplier count encoded bytes, Unicode code points or complete grapheme clusters? Also establish whether the field is rejected when too long or whether a prefix is delivered. Those choices determine what the receiving application can rely on.

RFC 3629, published in November 2003, describes UTF-8 encodings of one to four octets per scalar value. An octet is eight bits. Here, a code-point count means a count of Unicode scalar values, excluding surrogate code points.

A default extended grapheme cluster groups code points under segmentation rules. This article uses Unicode 17.0.0 and Unicode Standard Annex #29, revision 47, dated August 17, 2025. The default is an approximation of user-perceived characters; a tailored profile can change boundaries. Neither the default label nor a cluster count guarantees a particular glyph rendering or perfect segmentation for every language.

References: [R1][U1]

TXT-57: twelve bytes, five scalars and three default clusters

TXT-57 is an original fictional short-text delivery case. Its complete supplied text is é🇺🇸X. The exact sequence is U+0065, U+0301, U+1F1FA, U+1F1F8, U+0058: lowercase e, combining acute accent, regional indicator U, regional indicator S and uppercase X. The code-point sequence defines the case regardless of how a display presents it. The supplied representation is held fixed; no normalization is applied in the comparison.

Applying RFC 3629's encoding rules gives U+0065 as hexadecimal 65, U+0301 as CC 81, U+1F1FA as F0 9F 87 BA, U+1F1F8 as F0 9F 87 B8 and U+0058 as 58. Their lengths add to 1 + 2 + 4 + 4 + 1 = 12 bytes. There are five scalar values.

The Unicode 17.0.0 property file places U+0301 in Extend and both regional indicators in Regional_Indicator. Rule GB9 keeps the accent with the preceding e. The regional-indicator rules GB12 and GB13 pair indicators according to their preceding RI count; this supplied pair stays together. GB999 supplies the remaining breaks.

The resulting three default extended clusters are U+0065 U+0301; U+1F1FA U+1F1F8; and U+0058. These occupy three, eight and one bytes respectively. This is a derivation for the specified text and rules, rather than an observed application or model result.

References: [R1][U1][U2]

Four raw bytes fail encoding; three complete code points split the pair

The explicitly supplied four-byte raw prefix is hexadecimal 65 CC 81 F0. Its first three bytes encode e and the combining acute accent. The final F0 begins a four-byte UTF-8 encoding whose remaining bytes are absent. This raw prefix is therefore incomplete UTF-8.

A four-byte budget does not require that cut. TXT-57's first complete cluster already fits in three bytes. A service permitted to deliver a shorter complete prefix could leave the remaining byte unused. The invalidity belongs to the specified raw cut, rather than to every possible policy with a four-byte maximum.

Now consider the first three complete code points: U+0065 U+0301 U+1F1FA. They occupy 1 + 2 + 4 = 7 bytes, and every scalar encoding is complete. This prefix is valid UTF-8. It nevertheless ends between U+1F1FA and U+1F1F8, inside the second cluster of the original text.

Recounting clusters only after shortening can conceal the distinction. In the shortened string, the lone regional indicator reaches the end of text and forms a cluster. The output can thus have two clusters of its own while failing to retain the original second cluster. For a complete-original-cluster requirement, compare the stopping position with boundaries in the supplied input.

References: [R1][U1]

Two complete original clusters require four scalars and eleven bytes

The prefix retaining TXT-57's first two complete default extended clusters is U+0065 U+0301 U+1F1FA U+1F1F8. It stops after the regional-indicator pair and before U+0058. It contains four scalar values and occupies 1 + 2 + 4 + 4 = 11 bytes. Only the final one-byte X is omitted.

This prefix satisfies a requirement to retain those two original clusters. The seven-byte, three-code-point prefix does not satisfy that same requirement, despite its valid encoding. The four-byte raw prefix also fails, at an earlier boundary. A supplier response that reports only a byte total or successful decoding leaves the cluster-preservation question open.

The same arithmetic shows why the one-to-four-byte range for individual scalars cannot be reused as a byte allowance per cluster. TXT-57's regional-indicator cluster alone occupies eight bytes. There is no universally fixed number of UTF-8 bytes per cluster, so a cluster limit needs a separate byte constraint wherever the receiving interface has one.

References: [R1][U1][U2]

Make conflicting limits produce an agreed delivery decision

For TXT-57, a precise requirement could ask for the first two complete original default extended grapheme clusters under Unicode 17.0.0, UAX #29 revision 47, preserving the supplied scalar values and order. Its expected prefix is then explicit: the four listed scalars occupying 11 bytes. A supplier's use of a tailored profile would need to be declared for a comparison against that requirement.

Add a four-byte receiving limit and the requested two-cluster prefix no longer fits. The buyer must decide whether to permit the first complete cluster alone, occupying three bytes, or require rejection because two complete clusters are necessary. Leaving this choice implicit lets different offers satisfy different tasks.

An upper limit of at most two clusters also differs from a requirement to retain the first two clusters. Returning one cluster meets that upper limit while omitting required content in the stronger request. Where truncation is allowed, wording such as the longest unchanged prefix ending at an original cluster boundary within all stated maxima makes the intended retention rule clear.

Ask the application owner to identify the exact input at the limiting stage, its counted unit, any segmentation version or profile, the permitted omission and the delivered prefix or rejection. These are requirements for interpreting an offer, rather than a text-repair or configuration procedure. For this case, the scalar sequence and its original stopping boundary make the claimed result inspectable.

The related Unicode normalization article addresses which distinctions survive a transformation. The prompt-coverage article addresses whether required material reaches the model. Those remain separate questions: preserving complete clusters at this text boundary does not establish complete prompt coverage or a model-token count.

References: [U1]

Connect a separate equipment requirement to physical installation

If the infrastructure plan separately requires equipment mounting, IT cabling, labels and a physical installation record, Leviathan's Rack-and-stack installation service is the relevant scoping route. Use the related project inquiry with a nonconfidential equipment list, site, connection requirements and intended handoff.

The appointed application owner remains responsible for the text-limit decision. The physical engagement defines its own installation work and records.

References: [L1]

Primary sources

  1. [R1] RFC 3629: UTF-8, a transformation format of ISO 10646, November 2003Checked 2026-10-21. Sections 1, 3 and 4 define octets, scalar encoding lengths and complete UTF-8 byte sequences. TXT-57's hexadecimal encodings and prefix byte totals are derived from its explicit scalar values.
  2. [U1] Unicode Standard Annex #29, revision 47: Unicode Text Segmentation, Unicode 17.0.0Checked 2026-10-21. Defines the named default extended boundaries and tailoring scope. Rules GB2, GB9, GB12, GB13 and GB999 support the comparison; glyph rendering is outside that claim.
  3. [U2] Unicode Character Database 17.0.0: GraphemeBreakProperty.txtChecked 2026-10-21. The default Other assignment and the Extend range U+0300..U+036F and Regional_Indicator range U+1F1E6..U+1F1FF supply the case's relevant property values. Boundary decisions additionally require UAX #29's rules.
  4. [L1] Leviathan Systems: Rack-and-stack installationChecked 2026-10-21. Public service scope describes agreed equipment mounting, IT cabling, labels, physical checks and installation records for an authorized installation package.

Need a separate physical equipment installation?

Share a nonconfidential equipment, site, cabling and handoff outline to discuss the physical scope, requirements and availability.

Discuss a ProjectEmail Project Scope