@robocoder opened this issue on January 1st 2010

Support issue: users who upload (eg recursive ftp put) an already extracted Piwik folder. Files missing or corrupt.

Enhancement: - build script: generate file manifest with filesizes (and optional checksum) - installation/upgrade: iterate thru manifest and verify integrity of installation

@robocoder commented on January 1st 2010

Attachment: Standalone integrity test script testintegrity.php

@robocoder commented on January 1st 2010

Attachment: Sample (generated) manifest.inc.php (using piwik 0.5.4) manifest.inc.php

@mattab commented on January 1st 2010

Marking it as feature request. This is an interesting improvement (I've seen it in Gallery installer before), but can take many seconds to execute, and adds a bit of overhead on the release process. If someone wants to grab the task and build it, please let us know.

@robocoder commented on January 1st 2010

Change to build script:

  cp piwik/misc/How\ to\ install\ Piwik.html .
  cp piwik/misc/WebAppGallery/*.xml .

+ find . -printf '%s ' -exec md5sum {} \; >config/manifest
+ 
  zip -r piwik-$VERSION.zip piwik How\ to\ install\ Piwik.html  > /dev/null 2> /dev/null
@robocoder commented on January 1st 2010

Revised manifest generator for build script:

find piwik -type f -printf '%s ' -exec md5sum {} \; | fgrep -v 'manifest.inc.php' | sed '1,$ s/\([0-9]*\) \([a-z0-9]*\) *piwik\/\(.*\)/\t\t"\3" => array("\1", "\2"),/; 1 s/^/<?php\nclass Manifest {\n\tstatic $files=array(\n/; $ s/$/\n\t);\n}/' > piwik/config/manifest.inc.php

Timings from an (ancient) Athlon 1.4GHz test box: - Generating the manifest.inc.php took 11.117 seconds on a clean (svn export) folder. - Loading manifest.inc.php, iterating through the files, and running file_exists/file_size/md5_file took 0.541 seconds.

@mattab commented on January 1st 2010

For the UI part, I suggest that if the test fails, a red icon appears in the system checklist page. If it passes, it is just one more green item in the list (with required functions and librairies).

@robocoder commented on January 1st 2010

(In [1756]) refs #1097 - file integrity check (optional) added to Installation. A warning is given if manifest.inc.php or md5_file() don't exist.

Matt: please update the build script with the 'find' in comment:6 and then mark this ticket as fixed.

@robocoder commented on January 2nd 2010

(In [1757]) refs #1097 - exclude development environments; translate exception messages

more php docs

@robocoder commented on January 2nd 2010

(In [1758]) refs #1097 - reorganize code to better accommodate an integrity check after an upgrade (not yet implemented in CoreUpdater)

@mattab commented on January 2nd 2010

before marking as fixed, integrity check must happen during upgrades too. I will update the release script once this is code complete, nice work on this, this was a very frequent support issue.

@robocoder commented on January 3rd 2010

(In [1764]) refs #1097 - check file integrity after updates; simplify some messages (e.g., md5 sum)

@robocoder commented on January 3rd 2010

(In [1765]) refs #1097 - refactoring

@anonymous-piwik-user commented on January 4th 2010

I think this will be a really great feature. Thanks a real lot !

@mattab commented on February 19th 2010

Anthon, is the piwik code ready for this new feature? is the only thing left the one line update to the build script?

@robocoder commented on February 20th 2010

Matt: yes, it's ready.

While you're at it, please also add a line to the build script to remove unit test directories under the plugins folder.

  rm -rf piwik/tests/*
+ find piwik/plugins -name tests -type d -exec rm -rf {} \;
  mv README.txt piwik/tests/
@mattab commented on March 12th 2010

Anthon,

I added the find to delete the tests plugins folders and I get:

find: piwik/plugins/UsersManager/tests: No such file or directory
find: piwik/plugins/UserSettings/tests: No such file or directory
find: piwik/plugins/Actions/tests: No such file or directory
find: piwik/plugins/LanguagesManager/tests: No such file or directory
find: piwik/plugins/UserCountry/tests: No such file or directory
find: piwik/plugins/AnonymizeIP/tests: No such file or directory
find: piwik/plugins/SitesManager/tests: No such file or directory

is that expected?

@mattab commented on March 12th 2010

good stuff Anthon! closing

@robocoder commented on March 12th 2010

Yes, expected because 'find' is diving through folders while we remove it. We can suppress the messages using redirection, e.g, 2> /dev/null, or use xargs (if you don't mind the extra process being spawned), e.g.,

find piwik/plugins -name tests -type d -exec rm -rf {} \; 2>/dev/null

find piwik/plugins -name tests -type d -print0 | xargs -0 /bin/rm -rf
@mattab commented on March 18th 2010

(In [1944]) Refs #1097 adding explanation when failure occurs, inviting user to reupload the files

This issue was closed on March 18th 2010
Powered by GitHub Issue Mirror