Add paragraph, clause, sentence break functionality to Excel template and conversion logic (using existing FlexText XML tags)

[rulingAnts]

Make sure this is backward compatible:
Add a way to have checkboxes or some other UI element in the Excel template that indicates sentence and paragraph breaks (especially useful for things like Staged Event Videos that are not a single unified text at all, but are sort-of treated that way in this process).

Research FlexText XML code for those breaks and implement them in the conversion logic of the python app as well please. Thanks :)!

[rulingAnts]

Got this from a coworker. I asked him how the conversion and import into FLEx went. Here's his reply:
"Thanks, it was pretty good but it counted the whole thing as one sentence with each row being a new clause (so 1.1-1.60 or whatever). I manually changed to each video being its own "sentence ", so video #1 became 1.1-1.3, then video #2 became 2.1-2.2 and so on."

I can forward his conversion results (can't post it here because this is a public facing website and I can't share language data publicly without permission from the community, but I can send you a copy of his converted flextext file so you can see what the results he's talking about looked like in XML and imported into FLEx if you want to do a trial import into a dummy project).

FLExText format does have XML tags for paragraph, line, and phrase breaks and there is documentation on the SIL Global website available for that. Here's the best recommendations for that:
https://software.sil.org/fieldworks/help/technical-documents/ (it takes a little digging and fiddling around to find the most helpful one, but there is ONE PDF among these that is fairly detailed and exhaustive about the flextext format XML tags and how they're used. Trial and error also helps a bit--i.e. modify the text to be how you want it in FLEx, and then re-export from FLEx and look at the resulting XML. It'll add GUIDs that it would use to modify the existing text with changes you made, and our app SHOULDN'T retain or generate those, but the example resulting structure we can imitate in our conversion in order to correctly set up word, phrase, morpheme, etc tags and extra line breaks and numbering).

Also, FLExText format is apparently based on a EMELD standard proposed before, so looking at that proposal might help understand the big picture concept of how the various levels of tags/elements could produce the results we want in empty lines, numbering, line breaks and paragraph breaks in the baseline tab.

Here's the documentation for EMELD: https://aclanthology.org/U03-1008.pdf

[rulingAnts]

In the baseline tab in FLEx, you can hit enter for new lines that start with a new number (for example after entering line 1 or line 1.1, if you hit enter you get line 2 or 2.1). I typically use that to signal suspected sentence breaks in my text. shift+enter changes the sub-line number (like from 2.1 to 2.2). I usually use that for clauses. I'm also using empty lines (enter twice) to signal boundaries between different videos in the Staged Event Video sets. I'm not sure how any of that looks in the FLEXTEXT XML, but it would not be difficult to test.

[bplimley]

Previously, we interpreted a blank interlinear block in the Excel template as a paragraph break. Do you see this functionality as replacing that convention?

[rulingAnts]

Not sure I'm understanding your reply. The problem right now is we're getting section characters instead of line breaks and paragraph breaks in the baseline of imported texts. We want line breaks and paragraph breaks instead. Paragraph breaks should happen where the user has skipped a line in the Excel sheet (keep in mind line break is the equivalent of Shift+enter and paragraph break the equivalent of enter).

My suggestion is create a new text in FLEx, have a couple "shift+enter" lines with word glosses and free translations (dummy ones), and then a couple of "enter" lines (you'll see the difference when you do it) with word glosses and free translations. Then export that text as FLEXTEXT and compare its XML with the XML export our app produces and see what's different. Hopefully that way we can find out why we're getting section characters instead of line breaks.