This decouples the IMAP and Sqlite units from the Geary.Folder and
Geary.Account interface. Geary.EngineFolder will no longer deal
with generic local and remote folders, rather it is built knowing
it is dealing with IMAP.
This is necessary because of the IMAP engine changes coming, that
is dealing with servers that return server data out of order,
and even complete requests out of order.
This implements a background prefetch of all normalized messages
in the folder. It does not implement the chunked prefetch scheme
we discussed. It uses a simple priority scheme as well, only
pulling one message at a time, although it may be worthwhile
later implementing a pull-back system where the prefetch waits
until higher-priority traffic completes before continuing.
This incorporates much better duplicate detection in the local database, using both RFC-822 Message-ID as well as IMAP metadata (internaldate, RFC822 size) to determine if a message is already stored in the database. Very useful when a message is stored in multiple folders, or an already-downloaded message is returned to a folder it originated in (i.e. INBOX).
Also some minor fixes to listing email by EmailIdentifier which save a roundtrip to the server for certain edge cases.
Conversations now fully monitor the Folder for both additions and removals of messages.
This exposed a couple of bugs in the database code, which are fixed here as well. I also
used this opportunity to clean up some of the internal Conversations code, simplifying Node
management. #4333 will be a big boost here when implemented.
addressing.
Positional addressing is a nightmare for a lot of reasons, especially keeping positions
up-to-date as the Folder mutates. Now, positions are returned with the email but for
advisory reasons only, and do not keep up-to-date. It is expected that a client will use
positional addressing to "bootstrap" the application's data store, then use EmailIdentifier
addressing to traverse the Folder's contents.
Prior commit did not completely fix the problem in the case where a remote folder open took
a long time. This bulletproofs the solution, but does mean that there will be some
situations where FAST messages return with EmailLocations that radically change in the near
future. As the contract allows for any changes whatsoever, this is acceptable.
The entire database module now uses Transactions in order to guarantee atomicity of all
operations. However, due to limitations in SQLHeavy and its implementation of async, we
can't use it (and SQLite's) transactional models. This patch introduces a rather hamhanded
transactional model where the entire database is locked for each request using a
NonblockingMutex. This is the safest approach, but certainly not an optimal one. When
SQLHeavy's code is in place, hopefully we can rip this out with a minimum of fuss.
This adds a new flag when listing messages, FAST. This indicates that the caller wants
messages that are immediately available to the Folder, avoiding a round-trip to the server
(or even disk) if possible. Not all folders will support FAST, but it can be used (as it is
now in the client) to quickly populate the message list and then initiate a connection in
the background to get the straight dope.
This commit finishes the second half of #3793 by detecting when messages have been deleted
(or moved out of) an open folder and notifying the system of the change. The nonblocking
synchronization primitives have been beefed up and may be broken out to a separate library
some day.
This commit also introduces the ReplayQueue, which replays events that occur on the server
locally. This class will probably become much more important as time goes on, perhaps used
to store user events that are replayed on the server as well.
This commit represents the first half of this ticket, as it only monitors additions (new
emails) in the folder. A second commit will follow for monitoring removals (deleted emails)
in the folder.
The commit for #3851 changed the semantics of how messages are stored in the local cache,
and not all the code was properly updated to recognize that, particularly in EngineFolder.
This patch also introduces EmailIdentifier, which allows for a mail message to be fetched by
a unique identifier rather than its position in the folder list. This is much more
efficient and less error-prone, and means that Email objects don't have to be updated if
their position changes (although there are still some questions in that area). For IMAP,
this means being able to use the message's UID.
Caused by a valac bug set off by a generic function designed to convert a Gee List to a Vala
array. Rather than fight it I've removed the function and added to FetchCommand a
constructor to build the command using a Gee Collection.
The SQLite Geary.Folder implementation now uses reference semantics, so when it's reselected
the same Folder object is used and in-memory state is consistent. Also found an off-by-one
error when first querying an IMAP Folder that caused the most recent new email not to be
downloaded the first time. Fixed as well in this commit.
Needed to rethink storage strategies as I researched this and realized that a true scarce database -- where the database is sparsely populated both in columns and rows -- is not feasible due to IMAP's UID rules. The strategy now means that the database rows are contiguous from the highest (newest) message to the oldest *requested by the user*. This is a better situation than having to download the UID for the entire folder.
This commit adds support for IMAP-specific properties, of which UIDValidity is crucial toward completing #3805. The additional code is to integrate these tables into the SQLite Geary backend and to make sure this information is requested from the IMAP server.
NOTE: This commit changes the database schema. Old databases will need to be blown away before running.
Fields added. However, it would be nice to use formatting to separate them from the body of the message. This is not easy with Gtk.TextView/Gtk.TextBuffer; see https://bugzilla.gnome.org/show_bug.cgi?id=59390. This may push us to move to Webkit earlier rather than later.
Now supporting folder heirarchies. The client will now descend looking for subfolders. This task now opens up multiple outstanding requests to the Engine as well as exercises the database schema.
Closing this ticket opens the door to finishing #3692.
This commit introduces lazy loading of folder contents, which allows the Engine to report back email in chunks rather than all at once (which might require a round-trip to the server). This allows for the local database results to be returned to the caller right away while background I/O is occuring.
The code base is growing much faster than expected, faster than Shotwell it seems (not necessarily line count, but files and necessary organization of the library vs. Shotwell's initial flat directory). After some thought decided to move to a more standard Vala/GTK naming scheme of all lowercase with dashes for spaces starting with namespace (minus the "geary-", unless the class was in the topmost namespace). Three motivations:
1. Often confusing when working on code to see three "Folder.vala" in the gedit tabs: one IMAP, one SQLite, and one the interface definition.
2. This paves the way for waf integration, as right now we're held up using it because it barfs on projects with two files of the same name in different directories.
3. I find the CamelCase in the file browser becoming hard on the eyes, and this scheme seems a little more browsable.
2011-06-27 11:24:39 -07:00
Renamed from src/engine/sqlite/api/Folder.vala (Browse further)
