engine: Convert from SQLite FTS3/4 to FTS5 for full-text-search

Add SQL migration that drops the old FTS4 MessageSearchTable table,
re-create as a FTS5 table, clean up the column names a bit, and adds a
flags column so unread/starred queries can be made fast.

Define a SQLite FTS5 extension function `geary_matches()` to replace
the FTS3 `offsets()` function which no longer exists in FTS5, based on
Tracker's implementation.

Update code to FTS5 conventions (docid -> rowid, etc), use new column
names, populate and update the flags column as the email's flags
change, and use new match function for getting matching tokens.

Advanced searches are probably currently broken, these will be fixed
by subsequent commits.
This commit is contained in:
Michael Gratton 2020-11-04 00:55:13 +11:00 committed by Michael James Gratton
parent 435a5e90f4
commit 4fe0d92147
13 changed files with 266 additions and 63 deletions

View file

@ -158,8 +158,10 @@ web_extensions_dir = client_lib_dir / 'web-extensions'
# Ensure SQLite was built correctly
if not cc.has_header_symbol('sqlite3.h', 'SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER', dependencies: sqlite)
error('SQLite3 is missing FTS3 tokenizer support. Please compile it with -DSQLITE_ENABLE_FTS3.\n'
+ 'See https://bugzilla.gnome.org/show_bug.cgi?id=763203 for details.')
error('SQLite3 was not built with FTS3 support. See BUILDING.md for details.')
endif
if not cc.has_header_symbol('sqlite3.h', 'Fts5ExtensionApi', dependencies: sqlite)
error('SQLite3 was not built with FTS5 support. See BUILDING.md for details.')
endif
#