View RSS Feed

armpit

Updating MAME CHDs

Rate this Entry
My MAME CHDs have been a mess for years. I had a complete set at one stage, but the project moved from version 4 to version 5 several years ago and I never performed an update. Today I decided that I wanted a little green tick next to the set in clrmamepro and decided to convert what I had so I could start the ~500GiB torrent with some percentage already complete. Converting by hand would be a pain as there are so many of them and it turned out that at one stage I must have converted some of them manually, so I decided to automate the task.

The easiest way to achieve this I decided would be with a perl script that would walk through the directory containing the files recursively and convert each CHD that it encountered. As some were already converted, I would need to test them first and skip any that were already version 5. This was easy to achieve using perl with a minimum of modules needing to be used. The only modules I used are File::Copy for moving the original and converted files around, File::Stat so I could skip zero length files, and I used Capture::Tiny so I could capture the STDERR output from the chdman command in order to handle errors.

The script consists of two subroutines, the first being:

Perl Code:
  1. sub test_chd
  2. {
  3. my $file = shift or die "ERROR: Cannot test non-entity!\n";
  4. my $ver;
  5. print "----> Testing: $file\n";
  6.  
  7. my ( $stdout, $stderr ) = capture{
  8. system("$chdman info -i $file");
  9. };
  10.  
  11. return 0 if $stderr =~ /error/g;
  12.  
  13. foreach ( split(/\n/, $stdout) ) {
  14. if ($_ =~ /^File/) {
  15. $ver = $_;
  16. chomp $ver;
  17. $ver =~ s/.*:\s//;
  18. }
  19. }
  20.  
  21. return $ver;
  22. }


This subroutine simply takes a file path as an argument and then checks it using chdman. STDOUT and STDERR are captured using Capture::Tiny. A greedy regex then tests if STDERR contains the word 'error' and returns 0 if it does which informs the main loop that an error has occured.

The output from a successful chd check looks something like:

Code:
chdman - MAME Compressed Hunks of Data (CHD) manager 0.195 (unknown)
Input file:   /mnt/storage/torrents/pd/MAME 0.175 CHDs/gauntdl/gauntd24.chd
File Version: 4
Logical size: 2,287,467,520 bytes
Hunk Size:    4,096 bytes
Total Hunks:  558,464
Unit Size:    512 bytes
Total Units:  4,467,710
Compression:  zlib (Deflate)
CHD size:     1,602,514,166 bytes
Ratio:        70.1%
SHA1:         3e055794d23d62680732e906cfaf9154765de698
Data SHA1:    d6d9b15f3e20e3456431a6799aceeb2c0b4336aa
Metadata:     Tag='GDDD'  Index=0  Length=35 bytes
              CYLS:34367,HEADS:5,SECS:26,BPS:512.

So the line we are interested in is 'File Version: 4'. We iterate over this info line by line until we hit a regex match on the word 'File' and push the line into $ver. The variable is then cleaned up leaving us with the integer only which is then handed back to the main loop.

The second subroutine is:

Perl Code:
  1. sub convert_chd
  2. {
  3. my $file = shift or die "ERROR: Cannot convert non-entity!\n";
  4. my $retval = 0;
  5.  
  6. print "Converting: $file\n";
  7. unlink $tmpfile if -f $tmpfile;
  8.  
  9. my $stderr = capture_stderr{ qx{$chdman copy -i $file -o $tmpfile} };
  10. $retval = 1 if $stderr =~ /error/g;
  11.  
  12. return $retval;
  13. }


This takes the file path as an argument and then hands it chdman which performs the conversion storing the new file in the configured location. Again we use Capture::Tiny, this time we are only interested in capturing STDERR, so we use the capture_stderr method. We return 1 if our regex matches the word 'error' so the main loop can handle the failure.


The main loop of the script is simply a recursive walk through the given directory. When we come across a file with the extension '.chd' we start our testing. If the file has a zero length we skip it and move on to the next one, otherwise we pass it to the test_chd subroutine. If test_chd returns a 0, null, or non-integer value then we move on to the next file. If the return value is an integer we test that it is less than $want_ver ($want_ver = 5 in our case) we then pass the file to the convert_chd subroutine.

Once conversion is finished we pass the converted file to test_chd to ensure the conversion worked. If the conversion was a success we move the new file to the appropriate directory. If $keep_old is set to 1 then we append '.old' to the original file before we move the new one into place. If the conversion was not a success then we move on to the next file.

The whole script is as follows:

Perl Code:
  1. #!/usr/bin/perl
  2.  
  3. use strict;
  4. use warnings;
  5.  
  6. use Capture::Tiny qw/:all/;
  7. use File::Copy qw/move/;
  8. use File::stat;
  9.  
  10. my $chdman = "/usr/local/bin/chdman";
  11. my $tmpfile = "/tmp/out.chd";
  12.  
  13. my $want_ver = 5;
  14. my $keep_old = 0;
  15.  
  16. die "ERROR: No source dir supplied!" if not @ARGV;
  17.  
  18. my @dirs = (@ARGV);
  19. my %seen;
  20. my $chdver;
  21.  
  22. # Strip trailing slashes
  23. my $x = 0;
  24. foreach ( @dirs ) {
  25. $dirs[$x] =~ s|/\z||;
  26. $x++;
  27. }
  28.  
  29. sub test_chd
  30. {
  31. my $file = shift or die "ERROR: Cannot test non-entity!\n";
  32. my $ver;
  33. print "----> Testing: $file\n";
  34.  
  35. my ( $stdout, $stderr ) = capture{
  36. system("$chdman info -i $file");
  37. };
  38.  
  39. return 0 if $stderr =~ /error/g;
  40.  
  41. foreach ( split(/\n/, $stdout) ) {
  42. if ($_ =~ /^File/) {
  43. $ver = $_;
  44. chomp $ver;
  45. $ver =~ s/.*:\s//;
  46. }
  47. }
  48.  
  49. return $ver;
  50. }
  51.  
  52. sub convert_chd
  53. {
  54. my $file = shift or die "ERROR: Cannot convert non-entity!\n";
  55. my $retval = 0;
  56.  
  57. print "Converting: $file\n";
  58. unlink $tmpfile if -f $tmpfile;
  59.  
  60. my $stderr = capture_stderr{ qx{$chdman copy -i $file -o $tmpfile} };
  61. $retval = 1 if $stderr =~ /error/g;
  62.  
  63. return $retval;
  64. }
  65.  
  66. while ( my $pwd = shift @dirs )
  67. {
  68. opendir(DIR, "$pwd") or die "Cannot open $pwd: $!\n";
  69. my @files = readdir(DIR);
  70. closedir(DIR);
  71.  
  72. foreach my $file ( @files )
  73. {
  74. if ( -d "$pwd/$file" and ( $file !~ /^\.\.?$/ ) and not $seen{$file} )
  75. {
  76. $seen{$file} = 1;
  77. push @dirs, "$pwd/$file";
  78. }
  79. next if ($file !~ /\.chd$/i);
  80.  
  81. if ( stat("$pwd/$file")->size eq 0 ) {
  82. print "Skipping zero length file: $pwd/$file\n";
  83. next;
  84. }
  85.  
  86. $chdver = test_chd("$pwd/$file");
  87.  
  88. if ( defined $chdver )
  89. {
  90. next if not $chdver =~ /^-?\d+\z/;
  91. if ( $chdver eq 0 )
  92. {
  93. print "ERROR: chdman encountered an error reading: $pwd/$file\n";
  94. next;
  95. }
  96. }
  97. else {
  98. print "ERROR: test_chd failed to return a value.\n";
  99. next;
  100. }
  101.  
  102. if ( $chdver < $want_ver )
  103. {
  104. print ">> Version 4 CHD found.\n";
  105. my $r = convert_chd( "$pwd/$file" );
  106.  
  107. if ( $r eq 1 ) {
  108. print "ERROR: conversion failed for: $pwd/$file\n";
  109. next;
  110. }
  111.  
  112. my $test = test_chd($tmpfile);
  113.  
  114. if ( $test ne $want_ver )
  115. {
  116. print "Conversion failed!\n";
  117. }
  118. else {
  119. print "Conversion succeeded!\n";
  120. if ( $keep_old eq 1 )
  121. {
  122. print "Creating backup of original file.\n";
  123. move "$pwd/$file", "$pwd/$file.old";
  124. }
  125. else {
  126. unlink "$pwd/$file";
  127. }
  128. move $tmpfile, "$pwd/$file";
  129. }
  130.  
  131. }
  132. else {
  133. print ">> Conversion not needed.\n";
  134. }
  135. }
  136. }

Updated 03-20-2018 at 05:07 PM by armpit (Small fixes.)

Tags: None Add / Edit Tags
Categories
Programming

Comments

RSS RSS 2.0 XML MAP HTML
CSS Nginx MariaDB CentOS CSS XHTML