Incompatibility with Penango

[MikeDacre]

Hi,

You probably won't care about this, but I like being able to sign with both GPG and S/MIME - to give me options. Penango is a cool S/MIME extension for Gmail in Firefox, but it doesn't play nice with WebPG. If both are used at the same time, all of the newlines are removed from the GPG text, and the little WebPG encapsulation is not displayed.

Here is an image:

Thanks for all the great work! This is a really promising extension.

[kylehuff]

Well, we can't have that.

I can't promise a timeline for results, but I will certainly investigate the issue.

Are you using the version of WebPG located the Mozilla add-ons site (v0.9.3)? I'd just like to confirm if this is occurring with the latest development version of WebPG.

[MikeDacre]

Hi Kyle,

Thanks for the fast response! Glad to see you are interested.

I am using WebPG v0.9.4 in Firefox 26 on Mac OS X 10.9.1 with Penango 2.6.1.7. However, I just upgraded to v0.9.4 today, before that I had v0.9.3. I think the error may have been from the earlier version because I can't replicate it now, I am not 100% certain, because now Gmail in Firefox is very slow and Penango is not running properly - the UI elements it uses appear sporadically. I have a hunch that the error I reported was a problem for v0.9.3 and than now the issue is some other incompatibility between WebPG and Penango that is causing the slow down. I don't know for sure though.

I will test it tomorrow on my fast Linux machine at work, and I will try a clean Firefox profile too if you want. It may be worth testing it yourself though.

Thanks again for the great extension,

Mike

[kylehuff]

@MikeDacre

Great news (if v0.9.4 actually fixes that issue). In regards to the slowness and UI issues in GMAIL, I think that is not necessarily related to usage with Penango; I have other reports of slowness in GMAIL with firefox (it doesn't happen in Chrome), and as soon as I wrap up the PGP/MIME support in WebPG I will investigate that issue.

I will try to have some results for testing this weekend, but I can't promise anything with my schedule. I'll report back here when I have something to test out.

Thanks for your involvement in making WebPG better, it is greatly appreciated!

[kylehuff]

@MikeDacre

The most recent commit to the master branch should address the performance issue(s) with WebPG in GMAIL on Firefox; could you confirm this? (this "fix" may or may not address the UI issues you were experiencing... I would be interested to know that as well.)

[MikeDacre]

@kylehuff

Thanks for that, unfortunately I am having trouble building the extension, possibly as I have never built an extension before. I ran ./config_build.sh and then I had to edit ./build.sh and replace all instances of '--verbose' and ''--parents' with '-v' and '-p' respectively in order to get it to run. It ran fine then, but after install I couldn't get webpg to run, when I tried to open webpg preferences I got the following error:

Would it be possible for you to compile a version for me and put it somewhere I can get it. You can also email me if you prefer.

Thanks!

[kylehuff]

@MikeDacre

I'm sorry, I assumed you were using the development XPI -- I intended for you to use http://webpg.org/download/webpg-firefox/webpg-firefox-devel.xpi

As for the build script, I've never really tested the portability; I will look into as soon as I get a chance.

[MikeDacre]

@kylehuff

Thanks Kyle, that worked - I thought that you wanted me to use the git version :-) The performance issues are indeed solved, but unfortunately it looks like the UI issues are still there:

In addition, it seems that the gmail webpg button isn't there now:

This isn't an issue, as I can right click and use the 'Clearsign text' option.

[kylehuff]

Well, I think we are running into 1 of 2 issues (maybe both together?)

1.) When you sign via the context menu, the usual pre-send events for WebPG are not being called. (such as, setting the transfer mode, etc)

and/or

2.) Pendengo is parsing the input message in a manner that affects white-space and/or line-breaks (less likely, however possible)

Unfortunately, there isn't a whole lot I can do about either case. In the first case, I could have the right-click sign event set up the message for sane transfer if gmail is detected -- this could get tricky, though. I'll investigate that solution.

Any attempt to "correct" the second case would probably break the formatting for Pendengo -- which is equally as bad (if not worse) IMO.

As for the UI -- what appears to be going on here is that Pendengo and WebPG are both cloning the "SEND" button and hooking custom events to it. It appears that Pedengo might be removing the button entirely, which means WebPG cannot find it to make a copy. This is an "order of events" issue that I have little influence over. In reality, both WebPG and Pendengo are trying to do the same thing(s) in slightly different ways (or the same way), and it will inevitably will lead to issues.

Here is what I suggest as a plan of action for a potentially feasible workaround -

1.) Enable the WebPG toolbar for use in GMAIL (right-click context operations for signing and encryption are not entirely safe - the content could be read and/or altered prior to the operation by the website hosting the textarea - so I am trying to move away from that entirely)

2.) Ensure that the requisite setup is performed within gmail when actions such as sign/encrypt are used (providing that it will not interfere with Pendengo operation)

3.) Find the time to implement above steps (wait, this should probably be step one, but who else is counting?)

If the above is actually feasible and reaches the stated goal(s), one should be able to sign/encrypt with WebPG manually when using Pendengo, and it shouldn't require a terrible amount of work.

Thoughts or remarks?

[MikeDacre]

I think that you are probably right about the UI issue. I like the idea of implementing the toolbar in gmail, that would probably fix it. I also like the idea of having your secure editor available as an option for composing gmail emails.

As for the line spacing issue... I think there may be something else going on here. Specifically, the issue happens when in HTML mode in Gmail, which I usually disable, but which apparently was re-enabled in Firefox. When I compose a message in plain text mode, the message is shown correctly, although your inline formatting module does not run, so the result is:

I am sure you already know this, but if the message is composed in rich text (html) format, the result is two messages, the plain text version and the html version. In the plain text version though, the line 'Signed Message' portion of the signature is devoid of newlines, although the 'Signature' portion is still fine. This doesn't explain all of the display issues in Firefox though, as in those messages, all of the newlines were stripped from the 'Signature' portion also (maybe a newline character choice issue?).

Here is an example of a message composed in rich text format in Chrome (publically available extension, not dev version):

Message source:

MIME-Version: 1.0
Received: by 10.229.174.7 with HTTP; Mon, 27 Jan 2014 18:00:57 -0800 (PST)
Date: Mon, 27 Jan 2014 18:00:57 -0800
Delivered-To: mike.dacre@gmail.com
Message-ID: <CAPd9ww-tVZWX3h8Xc0sZ+E4+VOrENhq_ezSnYD0pCn8x10dg3A@mail.gmail.com>
Subject: Test
From: Mike Dacre <mike.dacre@gmail.com>
To: Michael D Dacre <mike.dacre@gmail.com>
Content-Type: multipart/alternative; boundary=001a113450a6f0d95f04f0fe316c

--001a113450a6f0d95f04f0fe316c
Content-Type: text/plain; charset=UTF-8

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Testing -----BEGIN PGP
SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAEBAgAGBQJS5w9YAAoJEM4WR9mTbQFGu0MP/1jUoE5Yi1DqWNFFaPHpsjne
lKBypQTJ2REMhZ+6QnDthwhUQ2HqH9BWIqH8pD6KDVO2xBjgSbv1ZbcJjcmeB2hO
xTHu7xXji079nEqbo+rcJ2IIQcl04vGkrDGXyoeTwdI7Kaigtm/DkcrANnurHbDy
JZWEWlJXhV9CcrK0Ltrdm/8slwpfFkuvsYB+CfFEgc+vapeNUBd2hL54fDgUxs+n
8zHjLM+1p1cIfgxltg34jAS2mrY5Rc5dAI5NCthhz050y+bxlt1dS6w5pQy2PnHs
YXQ1bPHGV1PlQE01DIcxzOxIExgd/euWYYogrOLkaaTDUgVV+v7/viY2KKTz64jS
IHzfDPRQWp+BrCVNiDPULuxMBDoibcYRrjW8uZPvTrPsTPFrKSPeqDi2zwK3LYcx
MPPkAoxY3+PsEt51jr/PUU5QHuKK+1xfFF6VSASJrJKjklMrPSEfLoo9pB0QXcCp
R8ypp1e3NzqV86PX2ns9/X1X6JV+5lPgrZfvM0t3hBMH9lJAzOPAMnqDSrRaux7K
G2iJbNm1HzI8PqL1JwsLfQFT2G+55y1lxiOVswI36l0nyvlS3SvZCtPUQPNGFes1
BzRU0a1rcGcnv40eMGyi8c6z+R2cC6l4uHq8wA7frdNmXier0VckYE/Vay78SnC3
Kqb3I7kSsSgEp/N8QN6r =lFU4 -----END PGP SIGNATURE-----

--001a113450a6f0d95f04f0fe316c
Content-Type: text/html; charset=UTF-8

<div dir="ltr">-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Testing
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJS5w9YAAoJEM4WR9mTbQFGu0MP/1jUoE5Yi1DqWNFFaPHpsjne
lKBypQTJ2REMhZ+6QnDthwhUQ2HqH9BWIqH8pD6KDVO2xBjgSbv1ZbcJjcmeB2hO
xTHu7xXji079nEqbo+rcJ2IIQcl04vGkrDGXyoeTwdI7Kaigtm/DkcrANnurHbDy
JZWEWlJXhV9CcrK0Ltrdm/8slwpfFkuvsYB+CfFEgc+vapeNUBd2hL54fDgUxs+n
8zHjLM+1p1cIfgxltg34jAS2mrY5Rc5dAI5NCthhz050y+bxlt1dS6w5pQy2PnHs
YXQ1bPHGV1PlQE01DIcxzOxIExgd/euWYYogrOLkaaTDUgVV+v7/viY2KKTz64jS
IHzfDPRQWp+BrCVNiDPULuxMBDoibcYRrjW8uZPvTrPsTPFrKSPeqDi2zwK3LYcx
MPPkAoxY3+PsEt51jr/PUU5QHuKK+1xfFF6VSASJrJKjklMrPSEfLoo9pB0QXcCp
R8ypp1e3NzqV86PX2ns9/X1X6JV+5lPgrZfvM0t3hBMH9lJAzOPAMnqDSrRaux7K
G2iJbNm1HzI8PqL1JwsLfQFT2G+55y1lxiOVswI36l0nyvlS3SvZCtPUQPNGFes1
BzRU0a1rcGcnv40eMGyi8c6z+R2cC6l4uHq8wA7frdNmXier0VckYE/Vay78SnC3
Kqb3I7kSsSgEp/N8QN6r
=lFU4
-----END PGP SIGNATURE-----
</div>

--001a113450a6f0d95f04f0fe316c--

In Chrome, this displays correctly still (somehow), but in Firefox it doesn't, it displays as the no-line-spacing version, although it actually removes all of the newlines in the 'Signature' portion also for some reason.

If I compose the message in plain text format though, it displays OK in Firefox, but your inline formatting module doesn't kick in:

It works great in Chrome however:

This formatting issue isn't directly related to Penango though... something else is going on. I tried disabling Penango and restarting, and it is still happening. I checked my own public key at http://michaeldacre.com/mike_dacre_pubkey.txt and that is still formatting correctly, but no gmail signatures seem to be showing, even with Penango disabled. Additionally, when I sign a messaged with WebPG in Firefox and click 'Send' nothing happens... the message just doesn't send. I feel like this is either a bug with my configuration, or an unrelated bug with the dev version of WebPG for Firefox.

So, ignoring that UI issue (assuming it is an unrelated bug or something I am doing wrong), I think the best steps are:

1. Implement the WebPG toolbar and secure editor in Gmail
2. Figure out why HTML messages are getting their newlines stripped... or just force plain text only when sending signed or encrypted messages

I can't test any more though until I/we figure out the Gmail display issues in Firefox without Penango. Is it possible this is a new bug in the latest dev version?

Thanks!

[kylehuff]

@MikeDacre

When I compose a message in plain text mode, the message is shown correctly, although your inline formatting module does not run...

I believe this is caused by one of the changes I made in the last commit -- if this is the same issue, if you go into that message a second time, it will render. I have since corrected this and it will be fixed in an upcoming commit.

This doesn't explain all of the display issues in Firefox though, as in those messages, all of the newlines were stripped from the 'Signature' portion also (maybe a newline character choice issue?).

Most of the newline issues has to do with the fact that Firefox does not have the "textContent" property for HTML elements, only "innerText", which does not respect line-breaks the way "textContent" does.

Here is an example of a message composed in rich text format in Chrome (publically available extension, not dev version)

The version of WebPG currently on the mozilla add-ons site has lots of white-space issues - most of those have been either resolved or significantly reduced in the latest development cycle (v0.9.4)

If I compose the message in plain text format though, it displays OK in Firefox, but your inline formatting module doesn't kick in

I believe this is the same issue I spoke about before, if you open the message a second time in the development version, it should render. Again, that should be fixed pending a commit.

Additionally, when I sign a messaged with WebPG in Firefox and click 'Send' nothing happens... the message just doesn't send. I feel like this is either a bug with my configuration, or an unrelated bug with the dev version of WebPG for Firefox.

Are you sure this with the dev version? If so, I would be interested in seeing the contents of the javascript console after an attempt to send.

So, ignoring that UI issue (assuming it is an unrelated bug or something I am doing wrong), I think the best steps are:

1. Implement the WebPG toolbar and secure editor in Gmail

Working on it.

1. Figure out why HTML messages are getting their newlines stripped... or just force plain text only when sending signed or encrypted messages

I know why this happens, and in the dev version it should be greatly reduced. As for forcing plain text when sending, that is one of the "connection setup" steps that should be happening in the dev version. I am able to send valid PGP messages in Firefox using WebPG v0.9.4-dev in both HTML and Plaintext mode without incident.

One future feature that will avoid 90% of this headache is that WebPG will be able to send PGP/MIME messages, whereby the signature is an attachment and the body of the message is not altered. However, this will be a problem with Pendengo, as it will require PGP/MIME messages to be sent via the WebPG SMTP module, since WebPG will need to alter the message headers. I will have to devise a solution for that (if at all possible).

I will reference this bug when I commit the fix for rendering, at which point the XPI referenced in my previous comments will contain the changes.