This commit finishes the second half of #3793 by detecting when messages have been deleted
(or moved out of) an open folder and notifying the system of the change. The nonblocking
synchronization primitives have been beefed up and may be broken out to a separate library
some day.
This commit also introduces the ReplayQueue, which replays events that occur on the server
locally. This class will probably become much more important as time goes on, perhaps used
to store user events that are replayed on the server as well.
This commit represents the first half of this ticket, as it only monitors additions (new
emails) in the folder. A second commit will follow for monitoring removals (deleted emails)
in the folder.
The commit for #3851 changed the semantics of how messages are stored in the local cache,
and not all the code was properly updated to recognize that, particularly in EngineFolder.
This patch also introduces EmailIdentifier, which allows for a mail message to be fetched by
a unique identifier rather than its position in the folder list. This is much more
efficient and less error-prone, and means that Email objects don't have to be updated if
their position changes (although there are still some questions in that area). For IMAP,
this means being able to use the message's UID.
Caused by a valac bug set off by a generic function designed to convert a Gee List to a Vala
array. Rather than fight it I've removed the function and added to FetchCommand a
constructor to build the command using a Gee Collection.
The SQLite Geary.Folder implementation now uses reference semantics, so when it's reselected
the same Folder object is used and in-memory state is consistent. Also found an off-by-one
error when first querying an IMAP Folder that caused the most recent new email not to be
downloaded the first time. Fixed as well in this commit.
It's vital that the message table always have a contiguous list of messages from the highest (newest) to the oldest requested by the user. If the user requests messages early enough that there would be a gap, the missing patch in the middle needs to be downloaded (UIDs, at least) to hold their position in the database. Prior commit downloaded too many; this change makes sure only messages not being requested by user but unavailable in the database are prefetched.
Needed to rethink storage strategies as I researched this and realized that a true scarce database -- where the database is sparsely populated both in columns and rows -- is not feasible due to IMAP's UID rules. The strategy now means that the database rows are contiguous from the highest (newest) message to the oldest *requested by the user*. This is a better situation than having to download the UID for the entire folder.
This commit adds support for IMAP-specific properties, of which UIDValidity is crucial toward completing #3805. The additional code is to integrate these tables into the SQLite Geary backend and to make sure this information is requested from the IMAP server.
NOTE: This commit changes the database schema. Old databases will need to be blown away before running.
Fields added. However, it would be nice to use formatting to separate them from the body of the message. This is not easy with Gtk.TextView/Gtk.TextBuffer; see https://bugzilla.gnome.org/show_bug.cgi?id=59390. This may push us to move to Webkit earlier rather than later.
This ticket enables this, however IMAP properties are not yet coming up from the local database yet, so this isn't technically complete. That work will be included in the commit for #3805, which requires those properties.
Problem was that the client fetches folder objects for the special folders (Inbox, Drafts, etc.) prior to fetching the full folder list and the Engine's fetch_folder_async() call wasn't prepared to handle fetches for folders not located in the local database.
This commit introduces the idea of a Personality, customizations for particular IMAP servers and services to allow the interface to configure itself for more natural use. Also, this commit has the app start in the Inbox, and an optimization was added that makes showing what's in the Inbox (at least, what's in the local cache) come up instantly.
GMime needs to be initialized prior to use. It looks like it builds some internal caches for encoding/decoding, leading to a null pointer fault when not initialized.
Now supporting folder heirarchies. The client will now descend looking for subfolders. This task now opens up multiple outstanding requests to the Engine as well as exercises the database schema.
Closing this ticket opens the door to finishing #3692.
This commit introduces lazy loading of folder contents, which allows the Engine to report back email in chunks rather than all at once (which might require a round-trip to the server). This allows for the local database results to be returned to the caller right away while background I/O is occuring.
The code base is growing much faster than expected, faster than Shotwell it seems (not necessarily line count, but files and necessary organization of the library vs. Shotwell's initial flat directory). After some thought decided to move to a more standard Vala/GTK naming scheme of all lowercase with dashes for spaces starting with namespace (minus the "geary-", unless the class was in the topmost namespace). Three motivations:
1. Often confusing when working on code to see three "Folder.vala" in the gedit tabs: one IMAP, one SQLite, and one the interface definition.
2. This paves the way for waf integration, as right now we're held up using it because it barfs on projects with two files of the same name in different directories.
3. I find the CamelCase in the file browser becoming hard on the eyes, and this scheme seems a little more browsable.
This is the first half of #3692, sorting the folders in the sidebar. Also, this patch ensures that messages are listed in chronologically descending order. (No option to change either sorts at this time.)
This completes the heavy lifting of persisting messages locally. The strategy is that the local database may be sparsely populated, both in the availability of messages in a folder and the fields of a message that is partially stored. As data is pulled from the remote server it's always stored in the database. Future requests will always go to the database first, preventing unnecessary network traffic.
Also, this patch will detect when a message is stored in multiple folders on the server. The database uses soft links from the folder to the message, so the message is stored only once in the database. This technique relies heavily on the availability and validity of the Message-ID header, but we expect this to be reliable the vast majority of the time.
This iteration now stores headers locally and fetches them first before going to the network. Work done in the database to deal with IMAPisms. More work on the GMime bindings (couple of mistakes in prior commit).
InternetAddress and its children and brethren were not properly bound by vapigen, probably because they aren't prefixed with "GMime" (unlike everything else in the library). Still some problems here, notably that certain private data members are exposed, but I've been unable to coax vapigen into not displaying them. Will work for now.
Much of the API between the local and net stores had to be reworked for consistency as well as planning ahead for how messages will be retrieved and stored efficiently. This work also attempts to keep in mind that other mail sources (POP, etc.) may be required in the future, and hopefully can be added without major rework.
Engine now uses a master Folder object that synchronizes between the network and the local store. Also, the Geary.Folder interface was getting ugly to code when the implementation was more than trivial, so moved to standard getters for it.
This large diff represents a growth of the architecture to persist IMAP data as its downloaded. When listing folders, a local database is consulted first to immediately feed to the caller. In the background, network calls fetch the "real" list. The two are collated for differences which are reported to the caller via signals, who are then responsible for updating the user interface appropriately. No other synchronization work is represented in this diff.
Note that this breaks functionality: when a folder is selected, no messages appear in the message list. Fixing this requires more work, and this patch was already large enough. It's ticketed here: #3741
--save-temps only speeds up compilation when valac is invoked for compile-only and C compilation and linking are done separately and in parallel. Since --save-temps leaves a lot of extraneous files sitting around that clutter "git status" (see #3690), removing it from the Makefile.
Keepalive could cause a segfault if the server closed the connection. This fixes that as well as updates some of the exception-throwing code to use more appropriate ImapErrors.
Message contents are now fetched from the server and displayed in the third pane when a message is selected. Also made session management a bit smarter via ReferenceSemantics.
The result of a SELECT or EXAMINE command is now parsed and returned to the caller. This information is boiled down to the Geary.Folder interface, which adds information about the folder to the object.
The ClientSessionManager maintains a pool of connections to an IMAP server, reusing them whenever possible. It allows for multiple folders to be managed and monitored by an application without having to change contexts constantly (which also introduces side-effects, esp. with expunging deleted messages).
geary now has an (unsorted) list of folders in its sidebar. When one is clicked on the messages in that folder are displayed. This patch also fixes an issue in the Serializer that wasn't dealing with quoted strings properly.
This adds a VAPI (including generation files and Makefile) for GMime to the repo, which we'll be using extensively to come. This VAPI is incomplete in many ways, so care should be used going forward.
Also, with GMime now interpreting RFC822 dates, can now pretty-print them. Prettier From: names also added this time around.
Client code will display a window listing Date, Subject, and From of all messages in inbox. Username and password must be specified as a command argument. Note that current IMAP envelope parsing doesn't handle group lists, which means some messages will be skipped over.
ClientSession can now automatically send NOOP commands as keepalives at a specified interval. Also, the CommandResponse decoders have been moved into their own directory (they will soon be fertile and multiply). More work ahead on the FetchResults object, which should be like NoopResults and completely encapsulate all the information returned from a FETCH.
