This fixes annoying, albeit legit addresses like "mike@vee.net
<mike@vee.net>" from being flagged.
Followup to commit b7eea857.
* src/engine/rfc822/rfc822-mailbox-address.vala (MailboxAddress): Don't
test name for spoofyness if it's not distinct. Add unit test.
Turns out the client was in inexplicably using
MailboxAddresses.to_rfc822_string() in a bunch of places to display
emails to users, mostly in the composer. This fixes that.
Followup to b7eea857.
* src/engine/rfc822/rfc822-mailbox-addresses.vala (MailboxAddresses):
Make list param for default ctor nullable for your testing convenience,
add a test. Add to_full_display() method that the client should nearly
always be using to display a list to users, use that in all the places.
Remove empty param for list_to_string since all but one callers just
pass in the empty string, which also fixes some stupid regressions
introduced in commit 178ce351 that causes it actually break
to_rfc822_string, rather than fix it as claimed, update call sites.
Both RFC mailbox address names and mailboxes/local-names may by RFC 2047
be Quoted-Printable or Base64 encoded. This patch ensures these parts are
correctly decoded when parsing a RFC 822 message, so that they are
displayed to the user in human-readable form.
Part 2 of Mailsploit mitigation.
* src/engine/rfc822/rfc822-message.vala (Message): Since GMime.Message's
convenience properties for accessing header values such as senders,
recipients, etc. in string form are presented as human-readable, not
RFC822 compliant strings, we can't re-parse them for use by this class
when it is being constructed from a GMime-based source. Instead,
iterate over all headers to get the raw values and parse those we are
interested in instead. Add unit tests.
* src/engine/rfc822/rfc822-mailbox-address.vala (BaseObject): Add gmime
constructor so we can handle construction and decoding from a GMime
InternetAddressMailbox object in a consistent way. Ensure both names
are decoded correctly, and mailboxes are decoded at all, from both
GMime and IMAP sources. If a GMime source's address has no @-symbol,
try to decode the whole thing first it in case the whole address is
encoded. Add unit tests.
* src/engine/rfc822/rfc822-mailbox-addresses.vala (MailboxAddresses):
Add append method and handle group addresses here instead of in Message
to simplify updated Message implementation.
* src/engine/rfc822/rfc822-message-data.vala (MessageData): Add append
method to simplify updated Message implementation.
* src/engine/rfc822/rfc822-mailbox-address.vala (MailboxAddress): Escape
and encode name and mailbox (local-part) when serialising as a RFC 822
string. Provide a means to get a RFC 822 version of the address only
for SMTP. Add unit tests.
* src/engine/smtp/smtp-request.vala (MailRequest): Use proper RFC 822
formatted version of mailbox addresses.
This adds a check for malware spoofing of RFC 822 mailbox addresses such
as those found in Mailsploit, and if found only displays the email
address part and not the mailbox name part.
Part 1 of Mailsploit mitigation.
* src/engine/rfc822/rfc822-mailbox-address.vala (MailboxAddress): Add new
is_spoofed method to check if the mailbox address looks like it has
been spoofed. Add is_distinct method to determine if the name and the
label is the same. Do whitespace and non-printing character stripping
when generating display versions of the mailbox address, rename methods
to make it more obvious what they do and update call sites. Add unit
tests to cover all this.
* src/client/conversation-viewer/conversation-message.vala
(ConversationMessage): Check name is distinct and is not valid before
displaying it. Use new MailboxAddress methods for getting display
versions of the address, to ensure we get the stripped versions of the
addresses.
* src/client/conversation-list/formatted-conversation-data.vala
(ParticipantDisplay): Ensure full addresses are always HTML-markup
escaped before displaying them as markup, to avoid dropping "<address>"
values as invalid HTML. Always show the full address if an address is
invalid.
* src/engine/util/util-string.vala (reduce_whitespace): Strip not only
whitespace but also non-printing characters. Add unit tests.
This generalises the approach used to execute the flag watcher and
background sync, provides a high-level means of managing local and remote
operations, and provides a means of compartmentalising operation-specific
code.
* src/engine/imap-engine/imap-engine-account-operation.vala
(AccountProcessor): Interface for denoting classes that implements some
account-specific operation.
* src/engine/imap-engine/imap-engine-account-processor.vala
(AccountProcessor): Class to manage the operation queue and execute
operations.
* src/engine/imap-engine/imap-engine-generic-account.vala
(GenericAccount): Create and manage an instance of AccountProcessor,
add queue_operation method to allow operations to be queued.
This allows us to compile the engine tests against the internal VAPI, so
we can unit test internal classes.
* CMakeLists.txt: Add top-level targets test-engine-run and
test-client-run that execute the respective test binaries. Make
the tests target depend on them and make them depend on their
binaries.
* Makefile.in: Add test-client and test-engin targets for convenience.
* src/CMakeLists.txt: Make the right invocation of valac to generate a
correct geary-engine-internal.vapi. Make gsettings schema generation
depend on geary-client so it exists for test-client if the main binary
hasn't been built.
* test/CMakeLists.txt: Split everything up into to builds, one for
test-engine and the other for test-client. Use geary-engine-internal
when building test-engine for access to internal classes and members.
* test/engine/util-idle-manager-test.vala,
test/engine/util-timeout-manager-test.vala: Use Glib MainContext rather
than the GTK main loop for pumping events so the test cases compile
without GTK.
* test/test-engine.vala, test/test-client.vala: New main source file for
test binaries based on old main.vala.
* src/engine/app/conversation-monitor/app-conversation-set.vala
(ConversationSet.add_email): Check to see if the email already exists
in the conversation, and if so merge its paths rather than just
returning.
(ConversationSet.merge_conversations_async): Simplify removing
conversations to be merged rather than manually handling it, meaning
email will be unconditionally removed rather than potentially being
conditionally removed via remove_all_emails_by_identifier.
(ConversationSet.remove_email_by_identifier): Simply update an email's
paths in the conversation if it has more than one, rather than removing
it.
* src/engine/app/conversation-monitor/app-conversation-set.vala
(ConversationSet.add_all_emails_async): Replace monitor arg with a
multiset of email paths, update call site to do the path lookup on the
caller's side instead. Add unit tests.
(ConversationSet.merge_conversations_async): Remove monitor arg with a
multiset of email paths, get paths for emails to be merged from the
conversations themselves, since they should be up to date.
(Conversation.remove_all_emails_by_identifier): Add unit tests.
* src/engine/app/app-conversation.vala (Conversation): We currently can't
easily unit test Conversation instances since the ctor requires a
ConversationMonitor argument, which is complicated. Replace the monitor
arg with a base folder (which is what it actually needs the monitor for
anyway) and remove the signals in favour of adding and modifying the
API to allow folder paths to be explicitly updated. Remove
clear_owner() method and dtor now it's useless. Update call sites. Add
unit tests to make sure the add/remove and multi-path related code
still works at least.
This allows us to compile the engine tests against the internal VAPI, so
we can unit test internal classes.
* CMakeLists.txt: Add top-level targets test-engine-run and
test-client-run that execute the respective test binaries. Make
the tests target depend on them and make them depend on their
binaries.
* Makefile.in: Add test-client and test-engin targets for convenience.
* src/CMakeLists.txt: Make the right invocation of valac to generate a
correct geary-engine-internal.vapi. Make gsettings schema generation
depend on geary-client so it exists for test-client if the main binary
hasn't been built.
* test/CMakeLists.txt: Split everything up into to builds, one for
test-engine and the other for test-client. Use geary-engine-internal
when building test-engine for access to internal classes and members.
* test/engine/util-idle-manager-test.vala,
test/engine/util-timeout-manager-test.vala: Use Glib MainContext rather
than the GTK main loop for pumping events so the test cases compile
without GTK.
* test/test-engine.vala, test/test-client.vala: New main source file for
test binaries based on old main.vala.
Add unit tests.
* src/engine/imap/transport/imap-deserializer.vala (Deserialiser): Add a
new EOL even handler to the FAILED state that is the one place where
the deserialize_failure() signal is fired and that any existing params
are reset.
(Deserialiser::push_line): Remove signal calls and unused return
value. Don't use EOS to detect NUL to avoid some confusion. Ensure an
EOL event is always fired so that the FAILED EOF handler is always run.
(Deserialiser::push_data): Remove signal calls and unused return value.
(Deserialiser::flush_params): Don't bother returning a state, since we
now always know what the next one should be when calling it. Only
trigger the parameters_ready() signal if there is a valid
parameter. Make logging consistent with the rest of the class.
(Deserialiser::reset_params): New common method for resetting any
existing deserialised message.
(Deserialiser::on_literal_char): Minor tidy up.
(Deserialiser::on_eos): Ensure that any existing param is sent on EOS
so that any BYE+EOS sent by the server is picked up.
Fixes Bug 781488.
* src/engine/imap/transport/imap-deserializer.vala (Deserializer): Update
char handling in START_PARAM state to be more explict and to handle all
possible chars, not just a subset. Then, simply defer to START_PARAM to
determine what to do when an invalid atom char is encountered when in
the ATOM state. Rename SYSTEM_FLAG state to just FLAG since it also
handles extension flags, and only use it for handling the first char in
the flag so "\*" can be treated appropriately, then defer to the ATOM
state, so flag keywords are treated in the same way as regular
atoms. Add unit tests.
* src/engine/util/util-html.vala (smart_escape): Don't bother attempting
to match a pair of tags, just something that looks like an opening tag,
since IMG is an empty element and it is valid to not have a closing
slash in HTML. Also, closing tags in HTML are optional anwyay. Add some
unit tests.
* src/engine/util/util-html.vala (smart_escape): Don't bother attempting
to match a pair of tags, just something that looks like an opening tag,
since IMG is an empty element and it is valid to not have a closing
slash in HTML. Also, closing tags in HTML are optional anwyay. Add some
unit tests.
* src/engine/api/geary-engine.vala (Engine::create_orphan_account): Fix
sense of test when determining the next account id to use. Add unit
tests.
(Engine::add_account): Made public so it can be used in public test.
This restores some old behaviour after teh WK2 port.
* src/client/conversation-viewer/conversation-message.vala
(ConversationMessage): Add alt_text param to ::save_image
signal. Replace ::set_action_param_string with
::set_action_param_string so we can pass a (url, alt_text) tuple for
the save_image tuple and update call sites. Set this tuple for the
action when enabling it, and pass the two values to the ::save_image
signal handler when activated.
* src/engine/api/geary-attachment.vala (Attachment::get_safe_file_name):
Add alt_file_name param method, use that when constructing a file
name. Add doc comment, test.
* src/client/application/geary-controller.vala: Updated to pass alt text
from save_imge signal emission all thr way through to
get_safe_file_name.
This ensures both inline images are saved using the specified content
filename, if any, and that an extension is attempted to be guessed when
no filename is specified.
Fixes Bug 713999, Bug 712923, Bug 778026.
* src/client/application/geary-controller.vala: Major rework of how
attachments are saved. Rework how save dialogs are constructed,
combining common code paths into one constrcutor method. Split up code
for saving one attachment vs many into two different methods. Ensure
all code baths ultimately use the same method to do the actual
saving. Lookup a attachment when saving an inline image and use that
by default. Get filenames from new Attachment::get_useful_filename
method that guesses if needed.
* src/client/conversation-viewer/conversation-message.vala
(ConversationMessage::save_image): Fix name of first param to reflect
what it actually is.
* src/engine/api/geary-attachment.vala (Attachment): Add
::get_safe_filename method that checks the type of both the
attachment's content and its file name, if any. Will construct an
appropriate file name if non is given. Add unit tests.
* src/engine/api/geary-email.vala (Email): Add new
::get_attachment_by_content_id to lookup attachments by MIME
content-id. Rename old ::get_attachment method to disambiguate.
Fixes Base64 encoded parts when fetching PREVIEW, and armour, etc
stripping etc not being applied to previews geneated via
Geary.RFC822.Message.
* src/engine/rfc822/rfc822-utils.vala
(Geary.RFC822.Utils::to_preview_text): Assume line endings are LF
encoded, update doc comment and unit tests to reflect that.
* src/engine/rfc822/rfc822-message-data.vala (PreviewText.with_header):
Add CRLF filter to preveiw text to strip CR chars from lines.
Bug 772607.
* src/engine/rfc822/rfc822-utils.vala (Geary.RFC822.Utils): Don't
truncate the preview here - fetch preview will already be truncated,
and the conversation message preview needs the full text anyway. Update
unit tests.
* src/client/conversation-viewer/conversation-message.vala
(ConversationMessage::ConversationMessage): Since we need to know if
the preview has been truncated so we know if we need to add an elipsis
or not, get the full preview and tuncate it here if needed.
* src/engine/rfc822/rfc822-message.vala (Message): Don't bother trying to
set the preview when creating a Geary.Email from the message. Minor
code & doc cleanup.
* src/engine/api/geary-email.vala (Email): Double length of max fetch
preview so HTML parts might actually pick up some content. Tidy up doc
comments.
* src/engine/rfc822/rfc822-utils.vala (to_preview_text): Don't bother to
check for MIME Content-* headers now that we are only treating a part
as plain text if it is actually text/plain.
Ensures that both appear the same to the user, and that the conversation
message preview gets the same armour and quote stripping that the fecth
preview does.
Added tests.
Bug 714317
* src/engine/rfc822/rfc822-utils.vala
(Geary.RFC822.Utils::to_preview_text): New common function to handle
generating a preview from a RFC 822 plain text or HTML string.
* src/engine/rfc822/rfc822-message-data.vala (PreviewText::with_header):
Move plain text armour and quote stripping to to_preview_text(), call
that to generate preview text.
* src/engine/rfc822/rfc822-message.vala (Message::get_preview): call
to_preview_text() to generate the preview text.
When generating the preview, only the first 128 bytes of the first MIME
part is fetched and used. If this part is text/html with a significant
amount of embedded CSS, then there is a good chance the string passed to
Geary.HTML::remove_html_tags() will be invalid, or be missing closing
elements. Since that function uses regexes that require balanced tags to
remove script and style blocks, then it was very possible that in these
cases this method will miss removing these blocks.
To solve this, remove_html_tags() is removed and its call sites are
replaced by calls to Geary.HTML::html_to_text(), which has been tidyied
up to produce more human-readable result.
Add unit tests to cover new html_to_text functionality and its call
sites.
* src/engine/util/util-html.vala: Remove remove_html_tags(). Update
html_to_text() to not just insert line breaks, but also insert spaces
and alt text, and ignore tags like HEAD, SCRIPT and STYLE, as
appropriate. Add an optional param to also allow skipping BLOCKQUOTE
elements, which we don't want in the preview.
