The NOOP keepalives were being every n seconds no matter what the
traffic was like. Now only being sent if no traffic has been
received after n seconds, meaning less traffic overall. Also, there
are now two different timers, one for selected mailbox and one for
authorized state, since receiving unsolicited responses only happens
when a mailbox is selected.
Also includes a small fix in Conversations where a nullable ref was
being treated as un-nullable.
This morning I discovered that Geary had crashed. When I attempted to rerun
it, it kept asserting in the Conversations logic. I discovered that there
was a problem with associating an email's Message-ID with its own Message-ID
processed in a prior email's References header field.
Turns out that there's a mailer out there that uses commas to separate
Message-IDs (a no-no in the spec). There's also a spam filtering mailer
out there that uses parentheses instead of angle brackets for its
Message-IDs (another no-no).
This commit strengthens the Message-ID parsing to handle these situations.
Unsolicited server data is how the server can notify of changes (adds, removes)
while the client is pulling data for an unrelated command. This patch catches
that unsolicited information and propagates it via signals.
Also some small fixes dealing with local/remote positional addressing in
EngineFolder and a bug that was introduced by the add/remove notifications in
MainWindow.
Problem was due to an internal private class of NonblockingBatch holding a weak ref
to its owner. Now holds a strong ref when scheduled and drops it when the operation
is completed, to prevent a reference cycle.
These changes are to speed up GenericImapFolder.prepare_opened_folder(), which is
the initial synchronization stage to ensure the local database is normalized against
the remote folder. The two big improvements are parallelization of database writes
(creates, removes) and skipping writes when unnecessary (if the message flags on
the server haven't changed, don't re-write them to the database).
Merging in additions to an email's fields in MessageTable could result in many
serialized SQLite operations that were independent of each other. This change
(a) parallelizes them via NonblockingBatch, and (b) reads the messages existing
fields and strips out writes to fields it already holds. This is legal because
none of the current fields in MessageTable are mutable on the server, but is not
allowable for ImapMessageProperties because they are.
Also, there were some naming inconsistencies in NonblockingBatch that are
corrected here.
Before we were fetching the entire message body (including attachments) to get the
preview text. This patch now offers the ability to fetch a small (128 byte) preview
of the email.
Also, since this ticket is about speeding up performance, I've introduced NonblockingBatch,
which allows for multiple async operations to be executed in parallel easily. I've added
its use in a few places to speed up operations, including one that was causing the lag
in #3799, which is why this commit closes that ticket.
Static member objects are not always initialized in Vala properly, causing this soft
assertion. Now using static properties to lazily create the various flags on demand.
This incorporates much better duplicate detection in the local database, using both RFC-822 Message-ID as well as IMAP metadata (internaldate, RFC822 size) to determine if a message is already stored in the database. Very useful when a message is stored in multiple folders, or an already-downloaded message is returned to a folder it originated in (i.e. INBOX).
Also some minor fixes to listing email by EmailIdentifier which save a roundtrip to the server for certain edge cases.
Conversations now fully monitor the Folder for both additions and removals of messages.
This exposed a couple of bugs in the database code, which are fixed here as well. I also
used this opportunity to clean up some of the internal Conversations code, simplifying Node
management. #4333 will be a big boost here when implemented.
addressing.
Positional addressing is a nightmare for a lot of reasons, especially keeping positions
up-to-date as the Folder mutates. Now, positions are returned with the email but for
advisory reasons only, and do not keep up-to-date. It is expected that a client will use
positional addressing to "bootstrap" the application's data store, then use EmailIdentifier
addressing to traverse the Folder's contents.
