Type Detection
How we identify data types automatically using pattern matching.
Detection Algorithm#
Our type detection uses a multi-step process:
1
Sampling
We sample up to 100 non-empty values from each column
2
Pattern Matching
Each value is tested against regex patterns for all 26 types
3
Scoring
We calculate the percentage of values matching each type
4
Selection
The type with the highest score above 50% is selected
Type Priority#
When multiple types match, we use a priority system. More specific types take precedence over generic types.
GenericSpecific
text
number
phone
url
email
Higher priority wins when confidence is similar
Supported Patterns#
Email Detection
Email Patterns
Standard: user@domain.com
Subdomains: user@mail.domain.co.uk
Plus addressing: user+tag@domain.comPhone Detection
Phone Patterns
US formats: (555) 123-4567, 555-123-4567, 5551234567
International: +1 555 123 4567, +44 20 7946 0958URL Detection
URL Patterns
With protocol: https://example.com/path
Without protocol: www.example.com
With query: example.com?foo=barOverriding Detection#
You can always override the detected type in the column configuration. This is useful when the automatic detection doesn't match your needs.
When to Override
Common scenarios include: ZIP codes detected as numbers, IDs detected as phone numbers, or date formats that don't match our patterns.