osis2mod very happily generates modules with no text visible

classic Classic list List threaded Threaded
47 messages Options
123
Reply | Threaded
Open this post in threaded view
|

osis2mod very happily generates modules with no text visible

Matěj Cepl
Hi,

trying to create an official version of the Czech Study Translation
(what used to be CzeKMS, but it is now severely outdated and
incomplete). I have original source in their custom XML scheme and I
have created XSLT stylesheet
(http://mcepl.fedorapeople.org/tmp/CEP2OSIS.xsl) which generates OSIS
file (http://mcepl.fedorapeople.org/tmp/bible.xml.gz 4MB). The OSIS file
validates and when I run

osis2mod modules/texts/ztext/czecsp/ bible.xml -v German

(osis2mod: $Rev: 2562 $; without -v parameter osis2mod core dumps;
judging by the backtrace on
http://mcepl.fedorapeople.org/tmp/osis2mod-gdb-output.txt.gz I would
assume some kind of endless loop happened).

However, when running with -v German, it ends up happily (with -d 1021
it generates tons of logs but I don't see any errors there;
http://mcepl.fedorapeople.org/tmp/osis2mod.log.gz 11.3MB). The problem
is that however I tried (including with diatheke and lookup build from
the same source as osis2mod) I don't see any text in the generated
bibles (./diatheke -b CzeCSP -f plain -e UTF8 -k Jn 3:16 returns
nothing; the ZIP file is http://matej.ceplovi.cz/cizi/CzeCSP.zip).

Anybody any ideas what's wrong?

Thanks in advance for any kicking in the right direction,

Matěj

P.S.: Also what is the state of the support for av11n (including -v
German) in AndBible and jSword?

_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

David Haslam
DMSmith is working towards making JSword capable to support av11n.
Only when that's in place can the front-ends based on JSword begin to take advantage thereof.

David
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

David Haslam
In reply to this post by Matěj Cepl
As a useful tip, it's generally a good idea to first to make a rawtext module with osis2mod.

This avoids some known bugs in making a ztext module directly using mod2osis.

You can easily use mod2zmod to compress it afterwards.

David

Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

David Haslam
In reply to this post by Matěj Cepl
The file bible.xml within http://mcepl.fedorapeople.org/tmp/bible.xml.gz seems to be not a text file but a binary!

In fact, it needs to be renamed as bible.xml.tar and decompressed yet again to get the real bible.xml

If you double compress something using tar then gzip, it's best to indicate that in the filename!

David
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

David Haslam
In reply to this post by Matěj Cepl
Your OSIS XML file makes extensive use of sub-identifiers.
This is the first time I've seen these in any OSIS file.
I have not seen these discussed in any previous threads in the mailing list.
This might turn out to be the real cause of why osis2mod doesn't produce a useful module.


The relevant section in the OSIS Reference Manual is:

15.3. Sub-identifiers

Translations also often split verses into parts, provided labels such as ‘a’ and ‘b’ for the separate parts.
Encoders may freely add sub-identifiers below the lowest standardized level. They are set off from the
standardized portion by the character ‘!’. For example:

Rev.2.20!b

Such subdivisions are not standard across different translations, so applications must be prepared to discard them when trying to locate a referenced location in a different edition.  These extensions are not considered a formal part of the canonical reference scheme, and so a work that uses them need not claim it is using a different scheme.


Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

David Haslam
In reply to this post by Matěj Cepl
If you make a temporary OSIS XML file with all the note elements removed, does that convert to a module with readable text?

If all the sub-identifiers happen to lie within the study notes, then that might be a profitable course of investigation.

David
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

David Haslam
In reply to this post by Matěj Cepl
The cause is much simpler!

Your path for module creation is in ztext, yet your .conf file is for a rawtext module.

Such a combination will always result in a module that appears to be empty of content.

btw. I've made the same mistake myself on rare occasions.

David
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

David Haslam
In reply to this post by Matěj Cepl
Though the OSIS validates, there is only one top level <div> element!

osis2mod expects each book to be a <div>, and each testament to be also a <div>

The <chapter ... tags should end with > rather than />

Also there are no </chapter> tags.

See http://crosswire.org/wiki/OSIS_Bibles#Body

David
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

David Haslam
It was easier for me to spot these issues in the body structure after I had used the XML Tools plugin for Notepad++ to first linearize the XML and then 'pretty print' the XML.

David
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

David Haslam
In reply to this post by David Haslam
What was the reason for not generating eID milestones for verses?

These should be present as per the example shown in the wiki page.

David
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

Matěj Cepl
In reply to this post by David Haslam
On 25.12.2011 22:57, David Haslam wrote:
> The file bible.xml within http://mcepl.fedorapeople.org/tmp/bible.xml.gz
> seems to be not a text file but a binary!
>
> In fact, it needs to be renamed as bible.xml.tar and decompressed yet again
> to get the real bible.xml

mitmanek:~ $ curl -L -O http://mcepl.fedorapeople.org/tmp/bible.xml.gz
   % Total    % Received % Xferd  Average Speed   Time    Time     Time
  Current
                                  Dload  Upload   Total   Spent    Left
  Speed
100 3708k  100 3708k    0     0   767k      0  0:00:04  0:00:04 --:--:--
  792k
mitmanek:~ $ file bible.xml.gz
bible.xml.gz: gzip compressed data, was "bible.xml", from Unix, last
modified: Sun Dec 25 07:43:38 2011, max compression
mitmanek:~ $ gunzip bible.xml.gz
mitmanek:~ $ file bible.xml
bible.xml: XML  document text
mitmanek:~ $ head <bible.xml
<?xml version="1.0" encoding="utf-8"?>
<osis xmlns="http://www.bibletechnologies.net/2003/OSIS/namespace"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xi="http://www.w3.org/2003/XInclude">
   <osisText osisIDWork="CzeKMS" osisRefWork="bible" xml:lang="cs"
canonical="true">
     <header>
       <revisionDesc resp="Matěj Cepl">
         <date>2010.09.07</date>
         <p>This is just an information about the book.</p>
       </revisionDesc>
       <work osisWork="CzeKMS">
         <title>Český studijní překlad Bible</title>
mitmanek:~ $

Sorry, that's your browser which did some repackaging behind the scenes.

Matěj

_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

Matěj Cepl
In reply to this post by David Haslam
On 26.12.2011 08:18, David Haslam wrote:
> The<chapter ... tags should end with>  rather than />
>
> Also there are no</chapter>  tags.

Meaning, <chapter/> couldn't be a milestone?

How does it align with this? (from
http://crosswire.org/wiki/OSIS_Bibles#OSIS_Milestones):

# It is allowable to use milestone elements for verses alone, or for
# both verses and chapters. The body example above is for the former.

Also, see the chapter 7.5 “The div element” (page 36) and example
therein in “OSIS Users Manual”
(http://www.bibletechnologies.net/utilities/fmtdocview.cfm?id=28871A67-D5F5-4381-B22EC4947601628B&method=title).
What I am missing?

Thanks for helping me.

Merry Christmass (and thoughtful feast of St. Stephen, let’s we all have
courage to stand up for what we believe!)

Matěj

_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

refdoc@gmx.net
On 26/12/11 19:55, Matěj Cepl wrote:
> On 26.12.2011 08:18, David Haslam wrote:
>> The<chapter ... tags should end with>  rather than />
>>
>> Also there are no</chapter>  tags.
>
> Meaning, <chapter/> couldn't be a milestone?

Chapters are usually milestoned. Nothing wrong with that.

Peter

_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

refdoc@gmx.net
The indeces (ot.vss, nt.vss) are essentially all 0. The osis file, while
validating contains not the correct info to create a module.

Peter

_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

refdoc@gmx.net
On 26/12/11 21:39, Peter von Kaehne wrote:
> The indeces (ot.vss, nt.vss) are essentially all 0. The osis file, while
> validating contains not the correct info to create a module.

The terminal output of osis2mod is meant to show what osis2mod is doing.
It does nothing.

The file is somehow not correct - even if it validates.

Peter

_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

refdoc@gmx.net
mod2imp shows that all content is put into verse 0 of each chapter.
Switching on headlines in xiphos makes a lot of text appear.

Peter



On 26/12/11 21:57, Peter von Kaehne wrote:

> On 26/12/11 21:39, Peter von Kaehne wrote:
>> The indeces (ot.vss, nt.vss) are essentially all 0. The osis file, while
>> validating contains not the correct info to create a module.
>
> The terminal output of osis2mod is meant to show what osis2mod is doing.
> It does nothing.
>
> The file is somehow not correct - even if it validates.
>
> Peter
>
> _______________________________________________
> sword-devel mailing list: [hidden email]
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page


_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

DM Smith-5
I hope to get a look at it tomorrow. Im busy w av11n stuff for jsword and entertaining my new grandson.

Cent from my fone so theer mite be tipos. ;)

On Dec 26, 2011, at 6:01 PM, Peter von Kaehne <[hidden email]> wrote:

> mod2imp shows that all content is put into verse 0 of each chapter.
> Switching on headlines in xiphos makes a lot of text appear.
>
> Peter
>
>
>
> On 26/12/11 21:57, Peter von Kaehne wrote:
>> On 26/12/11 21:39, Peter von Kaehne wrote:
>>> The indeces (ot.vss, nt.vss) are essentially all 0. The osis file, while
>>> validating contains not the correct info to create a module.
>>
>> The terminal output of osis2mod is meant to show what osis2mod is doing.
>> It does nothing.
>>
>> The file is somehow not correct - even if it validates.
>>
>> Peter
>>
>> _______________________________________________
>> sword-devel mailing list: [hidden email]
>> http://www.crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
>
>
> _______________________________________________
> sword-devel mailing list: [hidden email]
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page

_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

refdoc@gmx.net
On 27/12/11 00:33, DM Smith wrote:
> Im busy w av11n stuff for jsword

Peter von Kaehne likes that. A lot.

_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

refdoc@gmx.net
In reply to this post by refdoc@gmx.net
Ok Problems with the OSIS file so far:

1) Book divs are not there - see wiki
2) If you milestone chapters and verses you need to still "open" and
"close" them - by having a milestone with a sID and a milestone with a
eID. You have only the start milestones.

I am sure there is more, but try this for starters.

BTW the text contains an enormous number of references which are not
marked up. Once you got the above corrected please attend to this too.
It would make the module a lot better.

Yours

Peter



On 26/12/11 23:01, Peter von Kaehne wrote:

> mod2imp shows that all content is put into verse 0 of each chapter.
> Switching on headlines in xiphos makes a lot of text appear.
>
> Peter
>
>
>
> On 26/12/11 21:57, Peter von Kaehne wrote:
>> On 26/12/11 21:39, Peter von Kaehne wrote:
>>> The indeces (ot.vss, nt.vss) are essentially all 0. The osis file, while
>>> validating contains not the correct info to create a module.
>>
>> The terminal output of osis2mod is meant to show what osis2mod is doing.
>> It does nothing.
>>
>> The file is somehow not correct - even if it validates.
>>
>> Peter
>>
>> _______________________________________________
>> sword-devel mailing list: [hidden email]
>> http://www.crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
>
>
> _______________________________________________
> sword-devel mailing list: [hidden email]
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page


_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
Reply | Threaded
Open this post in threaded view
|

Re: osis2mod very happily generates modules with no text visible

Matěj Cepl
On 27.12.2011 01:48, Peter von Kaehne wrote:
> BTW the text contains an enormous number of references which are not
> marked up. Once you got the above corrected please attend to this too.
> It would make the module a lot better.

Yes, amount of notes and references is one of the reasons I really want
to make this to work.

However, I am not sure what you mean exactly by “references which are
not marked up”. Could you elaborate a bit, please?

Also, so far (I may relax this in the future) the OSIS file is generated
by a XSLT transformation from the original XML files used in translation
(https://gitorious.org/sword/czekms-csp_bible/blobs/master/CEP2OSIS.xsl). I
am not the best XSLT (try the other end of spectrum) so some things may
take some time to do. Maybe I rewrite the script in Python (although
XSLT works so far too well for me to be tempted) or I may relax this
requirement. We’ll see.

More after New Year!

Blessings on all of you,

Matěj

_______________________________________________
sword-devel mailing list: [hidden email]
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
123