[KPhotoAlbum] People and Locations categories not loaded anymore

[Johannes Zarl]

Hi Christoph,

it seems, that commit e7acc71a7abc4573653cedafbc514357971e9d47 breaks
loading the both categories People and Locations.
With 0eac65a931721c2373b95b5a49de20eee06be362 these both categories are
loaded.
To reproduce I created a minimalistic index.xml. If I load this with
commit e7ac... and click on "Places" there are 2 images assigned to
"None", with the commit before, Berlin and Munich are assigned to 1
image each.

Thanks for the report. I can confirm the issue on current master.

Is it possible that the speed optimized index.xml that I am using is not
converted correctly while migrating the old "standard"-categories?

I'll take a look immediately...

Cheers,
Johannes

Christoph
<?xml version="1.0" encoding="UTF-8"?>
<KPhotoAlbum version="5" compressed="1">
<Categories>
<Category name="Locations" icon="applications-internet"
show="1" viewtype="1" thumbnailsize="32" positionable="0">
<value value="Berlin" id="1"/>
<value value="Munich" id="2"/>
</Category>
<Category name="Folder" icon="folder" show="0" viewtype="0"
thumbnailsize="32" positionable="0"/>
<Category name="Media Type" icon="video" show="0" viewtype="0"
thumbnailsize="32" positionable="0"/>
</Categories>
<images>
<image file="1.jpg" label="0805106186"
startDate="2008-05-10T13:44:51" endDate="2008-05-10T13:44:51" angle="0"
md5sum="00273ab34b2b2755a6eed480c7d04a16" width="3072" height="2048"
Locations="1"/>
<image file="2.jpg" label="0805106186"
startDate="2008-05-10T13:44:51" endDate="2008-05-10T13:44:51" angle="0"
md5sum="00273ab34b2b2755a6eed480c7d04a16" width="3072" height="2048"
Locations="2"/>
</images>
<blocklist/>
<member-groups>
</member-groups>
</KPhotoAlbum>
_______________________________________________
KPhotoAlbum mailing list
https://mail.kdab.com/mailman/listinfo/kphotoalbum

[Tobias Leupold]

Hi Johannes/Christoph/List,

the problem is that XMLDB::Database::possibleLoadCompressedCategories looks
for the "new", already updated category names, while the XML still contains
the "old" ones. Thus, this data is lost.

I attached a patch fixing the issue, at least partially. I introduced a new
function to XMLDB::Database which tells the file version of the loaded
database. If it's too old (< 6), the "old" category names are looked up.

I'm pretty sure we still have to handle the case when localized standard
category names are stored in the database, which are substituted by their
modern C locale counterpart. This is a bit more tricky ;-)

But at least, I hope this is a good starting point for fixing this issue.

Cheers and good night, Tobias

[Johannes Zarl]

Hi Tobias,

Sorry, I didn't see that you were working on a solution, as well. In the
meantime, I have pushed my own solution, which can be seen in commit 015ccf0.

Could you please all check whether the fix in git master works for your
database correctly?

@Christoph: Sorry again for breaking your database! I don't know how this
could slip past me (especially since the bug affected my database as well!).

Johannes

Post by Tobias Leupold
Hi Johannes/Christoph/List,
the problem is that XMLDB::Database::possibleLoadCompressedCategories looks
for the "new", already updated category names, while the XML still contains
the "old" ones. Thus, this data is lost.
I attached a patch fixing the issue, at least partially. I introduced a new
function to XMLDB::Database which tells the file version of the loaded
database. If it's too old (< 6), the "old" category names are looked up.
I'm pretty sure we still have to handle the case when localized standard
category names are stored in the database, which are substituted by their
modern C locale counterpart. This is a bit more tricky ;-)
But at least, I hope this is a good starting point for fixing this issue.
Cheers and good night, Tobias

[Robert Krawitz]

Post by Johannes Zarl
Sorry, I didn't see that you were working on a solution, as well. In
the meantime, I have pushed my own solution, which can be seen in
commit 015ccf0.
Could you please all check whether the fix in git master works for
your database correctly?
@Christoph: Sorry again for breaking your database! I don't know how
this could slip past me (especially since the bug affected my
database as well!).

git master does *NOT* fix the problem. I loaded a known good database
and people and places were still missing.
--
Robert Krawitz <***@alum.mit.edu>

*** MIT Engineers Football -- Historic 9-0 2014, NEFC Champions ***
MIT VI-3 1987 - Congrats MIT Engineers 6 straight men's hoops tourney
Member of the League for Programming Freedom -- http://ProgFree.org
Project lead for Gutenprint -- http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton

[Tobias Leupold]

Post by Robert Krawitz
git master does *NOT* fix the problem. I loaded a known good database
and people and places were still missing.

fa38d2d71911409c73d1adf1f0fd22e66a88fb7e should also fix the problem with un-
compressed old databases. I Attached my testing database.

Please check if it works now and thanks for the bug report :-)

[Robert Krawitz]

Post by Tobias Leupold
Please check if it works now and thanks for the bug report :-)

It works, but then when I next load it I get the following error:

"An error was encountered on line 640, column 52: Expected to read
option, but read value."

The line in question is:

<value value="Picture frame 2"/>

and the column is the closing angle bracket.

The image in question is:

<image file="wedding/crop0001.jpg" label="crop0001" startDate="1996-05-26T00:00:00" endDate="1996-05-26T00:00:00" angle="0" md5sum="e3a06db191c163ff428da10746f7aca9" width="2810" height="1031">
<options>
<option name="Keywords">
<value value="Picture frame 2"/>
<value value="Picture frame"/>
<value value="Wedding"/>
</option>
<option name="Places">
<value value="Brookline, MA"/>
</option>
</options>
</image>
--
Robert Krawitz <***@alum.mit.edu>

*** MIT Engineers Football -- Historic 9-0 2014, NEFC Champions ***
MIT VI-3 1987 - Congrats MIT Engineers 6 straight men's hoops tourney
Member of the League for Programming Freedom -- http://ProgFree.org
Project lead for Gutenprint -- http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton

[Johannes Zarl]

Should be fixed in git master b9f0378. The database file was alright, but the
category name lookup was missing a fallback.

Johannes

Post by Robert Krawitz

Post by Tobias Leupold
Please check if it works now and thanks for the bug report :-)

"An error was encountered on line 640, column 52: Expected to read
option, but read value."
<value value="Picture frame 2"/>
and the column is the closing angle bracket.
<image file="wedding/crop0001.jpg" label="crop0001"
startDate="1996-05-26T00:00:00" endDate="1996-05-26T00:00:00" angle="0"
md5sum="e3a06db191c163ff428da10746f7aca9" width="2810" height="1031">
<options>
<option name="Keywords">
<value value="Picture frame 2"/>
<value value="Picture frame"/>
<value value="Wedding"/>
</option>
<option name="Places">
<value value="Brookline, MA"/>
</option>
</options>
</image>

[Robert Krawitz]

Post by Johannes Zarl
Should be fixed in git master b9f0378. The database file was
alright, but the category name lookup was missing a fallback.

Works.

What's the difference between the compressed and uncompressed format,
anyway?

Post by Johannes Zarl

Post by Robert Krawitz

Post by Tobias Leupold
Please check if it works now and thanks for the bug report :-)

"An error was encountered on line 640, column 52: Expected to read
option, but read value."
<value value="Picture frame 2"/>
and the column is the closing angle bracket.
<image file="wedding/crop0001.jpg" label="crop0001"
startDate="1996-05-26T00:00:00" endDate="1996-05-26T00:00:00" angle="0"
md5sum="e3a06db191c163ff428da10746f7aca9" width="2810" height="1031">
<options>
<option name="Keywords">
<value value="Picture frame 2"/>
<value value="Picture frame"/>
<value value="Wedding"/>
</option>
<option name="Places">
<value value="Brookline, MA"/>
</option>
</options>
</image>

--
Robert Krawitz <***@alum.mit.edu>

*** MIT Engineers Football -- Historic 9-0 2014, NEFC Champions ***
MIT VI-3 1987 - Congrats MIT Engineers 6 straight men's hoops tourney
Member of the League for Programming Freedom -- http://ProgFree.org
Project lead for Gutenprint -- http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton

[Tobias Leupold]

Post by Robert Krawitz
What's the difference between the compressed and uncompressed format,
anyway?

Just look at the two example files I sent to the list, there, you can see the
difference. The way tags are added to images is less complex (imo more
convenient) in the "fast" format, and IDs are used instead of the complete
names. Also, the tags go in an attribute of the image. Like:

<image ... Category="1,5,15"/>

For the "readable" format, the same would be:

<image ...>
<options>
<option name="Category">
<value value="Tag 1"/>
<value value="Tag 5"/>
<value value="Tag 15"/>
</option>
</options>
</image>

It's much more text to write to a file (per image!) and thus slower to
process. One can argue about which one is more "readable" ;-) but at least,
the "readable" format contains all information for each image inside the
<image> tag, so it would be easier to parse it using something else than KPA.
But it also contains a lot of redundant data.

I personally would rather develop a completely new XML format :-P but very
likely, this will not happen ;-)

[Johannes Zarl]

Post by Robert Krawitz

Post by Johannes Zarl
Should be fixed in git master b9f0378. The database file was
alright, but the category name lookup was missing a fallback.

Works.

Thanks for the feedback.

Post by Robert Krawitz
What's the difference between the compressed and uncompressed format,
anyway?

Tobias wrote most of it already, but the "executive summary" is:

From a user-perspective, the compressed format has the benefit of faster
load/save times.

The uncompressed format OTOH is well-defined XML (in the sense that you could
write an XML schema for it).

Cheers,
Johannes

[Christoph Moseler]

Great, works for me as well.

Thanks for the prompt fix.

Cheers,
Christoph

PS.: Johannes, don't worry about my database, of course I had a backup

Post by Johannes Zarl
Should be fixed in git master b9f0378. The database file was alright, but the
category name lookup was missing a fallback.
Johannes
_______________________________________________
KPhotoAlbum mailing list
https://mail.kdab.com/mailman/listinfo/kphotoalbum

[Tobias Leupold]

Post by Christoph Moseler
Thanks for the prompt fix.

Sorry for the mess ;-)

[Tobias Leupold]

Post by Johannes Zarl
Could you please all check whether the fix in git master works for your
database correctly?

I can't reproduce the problem with current git master and a compressed
database anymore. I attached my minimal testing database for convenience.

Post by Johannes Zarl
git master does *NOT* fix the problem. I loaded a known good database
and people and places were still missing.

Probably a problem with the "readable" db format. I'll investigate this!

Anyway, it's nice that we found this _before_ we did a release ;-)

[Tobias Leupold]

This patch should also work for databases with localized standard category
names.

[Robert Krawitz]

This patch should also work for databases with localized standard category names.

No, still losing the data.
--
Robert Krawitz <***@alum.mit.edu>

*** MIT Engineers Football -- Historic 9-0 2014, NEFC Champions ***
MIT VI-3 1987 - Congrats MIT Engineers 6 straight men's hoops tourney
Member of the League for Programming Freedom -- http://ProgFree.org
Project lead for Gutenprint -- http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton