Aggregate collaborative launch result hashes in the issue

[jorisstrakeljahn]

This PR adds a GitHub Action that keeps one summary comment on open collaborative launch issues, with a table of matching result hashes (the part people used to tally by hand). Discussed in #49 and maflcko's sketch in DrahtBot#80.

What

On issues titled Collaborative Launch …, the action creates a summary comment when the issue opens and edits that same comment when the table changes.

Participants report results with lines like:

HASH: <sha256 of final_result.txt>

Every distinct hash a user posts counts once for them, whether the lines sit in one comment or spread across several (voting rule). Reposting the same hash adds no votes. This covers results from multiple machines.
The summary table is sorted by match count:

hash	users	share of matches
xyz…	fjahr, jorisstrakeljahn	2/3
abc…	jorisstrakeljahn	1/3
def…	third-user	1/3

The denominator is the number of users who reported at least one hash. Comment edits and deletions trigger a rescan. Closed issues are ignored.

Accepted HASH: lines

Example	OK?
HASH: <sha256>	yes
HASH:<sha256> (no space)	yes
Hash: <sha256> / hash: <sha256>	yes
HASH: `<sha256>`	yes
HASH: **<sha256>**	yes
leading/trailing spaces on the line	yes
several such lines in one comment (distinct hashes)	yes, one vote per distinct hash
same hash repeated by the same user	counts once

Not accepted

Example	OK?
wrong length / non-hex	no
HASH: with no hash	no
hash only in prose/logs, no HASH: prefix (e.g. The SHA-256 hash of the result file is: …)	no
> HASH: <sha256> (quoted line)	no
comment containing <!--meta-tag:bot-skip-->	whole comment ignored

Suggested addition for launch issue bodies (for example #63):

### Reporting your result

Post the SHA-256 of your `final_result.txt` on its own line:

HASH: <sha256 of final_result.txt>

If you ran on multiple machines, report each result. Every distinct hash you post counts once for you. A bot keeps a summary table in a comment below.

Demo (fork)

• #3: full test round on the current version
• Earlier rounds against the previous version: #1, #2

After merge

#63 is already open for the 6 August launch. Merge alone will not post the summary there (the issue was not newly opened). If this lands before the launch, someone with write access can trigger a one-time refresh (Actions: Aggregate Collaborative Hashes, issue number 63, or a small issue edit). Otherwise the first comment on #63 after the merge creates the summary (empty table until HASH: lines arrive); later comments update it.

Happy to take suggestions on the summary comment or any other stuff.

[jorisstrakeljahn]

One note on the summary format: the table matches the example from #49 / DrahtBot#80, plus two extra lines below it (Participants: and Leading:). Both can be read from the table itself, so they are easy to drop if a minimal summary is preferred.

[fjahr]

I just tried it in both of your example pulls and it does not seem like the tables are getting updated. It seems like it already didn't work with the second hashes you posted there.

[fjahr]

We don't need documentation on this in the readme. We should rather keep a comment on this in the bottom of the collaborative run issues. But since it's environment dependent it shouldn't go in the readme.

[jorisstrakeljahn]

okay, will remove that

[fjahr]

plus two extra lines below it (Participants: and Leading:)

I would prefer it if the table was sorted instead, i.e. the leading hash(es) is at the top. And then there can be a third row with the share of the total matches.

Something like this:

hash	users	share of matches
xyz	@fjahr, @jorisstrakeljahn	2/3

[jorisstrakeljahn]

I just tried it in both of your example pulls and it does not seem like the tables are getting updated. It seems like it already didn't work with the second hashes you posted there.

Saw that, you wrapped the hashes in backticks. Right now only a bare line is recognized, so HASH: ... works and
HASH: '...' does not.

Perhaps you could try it again without doing that. Of course, I’m also happy to adjust the rule so that hashes enclosed in `` are also taken into account.

[fjahr]

Seemed to work but there are still some issues with multiple results, I noted them in the demo issues.

I’m also happy to adjust the rule so that hashes enclosed in `` are also taken into account.

Yeah, that should definitely be adjusted. I prefer the use of backticks but it should not be a requirement. Any formatting of the hash (even bold etc.) should be ignored just like different captializations of hash/Hash/HASH

[fjahr]

I am also getting email notifications that the bot posted a new comment. We don't get these for drahtbot when it updates the acks table. this bot also shouldn't trigger notifications as that is too noisy and the updated corresponds to a user comment anyway.

[jorisstrakeljahn]

I would prefer it if the table was sorted instead, i.e. the leading hash(es) is at the top. And then there can be a third row with the share of the total matches.

The table is already sorted with the leading hash at the top. Happy to switch to your format though: drop the Participants: / Leading: lines and use a share of matches column like 2/3 instead of the plain count.

[fjahr]

Happy to switch to your format though: drop the Participants: / Leading: lines and use a share of matches column like 2/3 instead of the plain count.

Yes, the leading also doesn't really work because it just shows one hash even when there are multiple with the same count of matches:

Leading: 4b715c80253505f119c171e2eae798c54977702b3b4ae0f7ae0ba337b649b02c (1/2)

[jorisstrakeljahn]

Worked here now as well but the second result should not be ignored. People post multiple hashes from different machines, that should still be taken into account
(demo #1)

@fjahr quick check on the voting rule, because this conflicts a bit with what I implemented from #44 (one vote per participant, not per machine).

Right now the bot keeps one result per GitHub user and only the first HASH: line in that comment. So in your test comment only d9f95adb… counts and 35a4b28f… is dropped. Same idea as ignoring later HASH: comments from the same user.

Do you want:

• still one vote per user, but consider all HASH: lines they posted and count them toward the most common hash among those (as in your #44 note), or
• count every HASH: line as its own match, even from the same user?

[fjahr]

(one vote per participant, not per machine)

It is one vote per user but they can vote for multiple hashes, not multiple times for the same hash, which would be one vote per machine. So it's neither of what you wrote. You take every HASH result from them into account, can be multiple lines in one comment or across multiple comments but they can vote for each hash once at most.

[jorisstrakeljahn]

Got it: every distinct hash a user posts counts once for them, duplicates of the same hash don't.

I'll rework the bot with the review notes and open a fresh demo issue when it's up!

[jorisstrakeljahn]

Reworked the action based on the review and ran a full test round in demo issue #3. Changes:

• Voting follows this rule: every distinct hash a user posts counts once for them, across any number of lines and comments. Reposting the same hash adds no votes.
• The parser accepts backticks, bold and any capitalization of hash: (comment). Quoted lines (> HASH: …) and hashes inside logs are still ignored.
• The table is sorted by matches and uses the suggested format, hash | users | share of matches with counts like 2/3. The Participants: and Leading: lines are gone (comment, comment).
• Usernames are rendered as plain links instead of @-mentions, and the bot only edits its single summary comment when the table changed, so it should no longer send notification mails (comment).
• Deleting a comment now triggers a rescan and updates the table (comment).
• The README section is removed (review comment).

Demo issue with these changes here: #3

[fjahr]

ACK 732d6aa

Tested all the edge cases I could think of. Looks good to use for Thursday.