Merging in additions to an email's fields in MessageTable could result in many
serialized SQLite operations that were independent of each other. This change
(a) parallelizes them via NonblockingBatch, and (b) reads the messages existing
fields and strips out writes to fields it already holds. This is legal because
none of the current fields in MessageTable are mutable on the server, but is not
allowable for ImapMessageProperties because they are.
Also, there were some naming inconsistencies in NonblockingBatch that are
corrected here.
Before we were fetching the entire message body (including attachments) to get the
preview text. This patch now offers the ability to fetch a small (128 byte) preview
of the email.
Also, since this ticket is about speeding up performance, I've introduced NonblockingBatch,
which allows for multiple async operations to be executed in parallel easily. I've added
its use in a few places to speed up operations, including one that was causing the lag
in #3799, which is why this commit closes that ticket.
This incorporates much better duplicate detection in the local database, using both RFC-822 Message-ID as well as IMAP metadata (internaldate, RFC822 size) to determine if a message is already stored in the database. Very useful when a message is stored in multiple folders, or an already-downloaded message is returned to a folder it originated in (i.e. INBOX).
Also some minor fixes to listing email by EmailIdentifier which save a roundtrip to the server for certain edge cases.
Conversations now fully monitor the Folder for both additions and removals of messages.
This exposed a couple of bugs in the database code, which are fixed here as well. I also
used this opportunity to clean up some of the internal Conversations code, simplifying Node
management. #4333 will be a big boost here when implemented.
This adds support for retrieving partial header and body blocks straight from the email, and
therefore support to pull the References header from a message (which, for some reason, IMAP
doesn't support or include in the FETCH ENVELOPE command). This is necessary for email conversations (#3808).
This required a change to the database schema, meaning old databases will need to be blown
away before starting.
The entire database module now uses Transactions in order to guarantee atomicity of all
operations. However, due to limitations in SQLHeavy and its implementation of async, we
can't use it (and SQLite's) transactional models. This patch introduces a rather hamhanded
transactional model where the entire database is locked for each request using a
NonblockingMutex. This is the safest approach, but certainly not an optimal one. When
SQLHeavy's code is in place, hopefully we can rip this out with a minimum of fuss.
Needed to rethink storage strategies as I researched this and realized that a true scarce database -- where the database is sparsely populated both in columns and rows -- is not feasible due to IMAP's UID rules. The strategy now means that the database rows are contiguous from the highest (newest) message to the oldest *requested by the user*. This is a better situation than having to download the UID for the entire folder.
The code base is growing much faster than expected, faster than Shotwell it seems (not necessarily line count, but files and necessary organization of the library vs. Shotwell's initial flat directory). After some thought decided to move to a more standard Vala/GTK naming scheme of all lowercase with dashes for spaces starting with namespace (minus the "geary-", unless the class was in the topmost namespace). Three motivations:
1. Often confusing when working on code to see three "Folder.vala" in the gedit tabs: one IMAP, one SQLite, and one the interface definition.
2. This paves the way for waf integration, as right now we're held up using it because it barfs on projects with two files of the same name in different directories.
3. I find the CamelCase in the file browser becoming hard on the eyes, and this scheme seems a little more browsable.
2011-06-27 11:24:39 -07:00
Renamed from src/engine/sqlite/MessageTable.vala (Browse further)
