A PDF arrives, a cron job checks once a minute if new files have arrived, it finds the file and starts a little processing chain.<\/p>\n\n\n\n
### Process the file ###\n\n # OCR the document\n ocrmypdf -l deu \"$entry\" \"$OUTPUT_DIR${entry##*\/}\"\n # extract all text of the pdf to a text file\n pdf2txt -o \"$OUTPUT_DIR${entry##*\/}.txt\" \"$OUTPUT_DIR${entry##*\/}\"\n # save thumbnails of the pages of the pdf\n convert \"$entry\" -quality 30 \"$OUTPUT_DIR${entry##*\/}.jpg\"<\/code><\/pre>\n\n\n\nWhat I am using is OCRMyPDF to OCR the incoming file, it writes a new output file that has text underlay.<\/p>\n\n\n\n
With pdf2txt the text is exported and all blanks and empty lines are removed, as to have one string of text for a file.<\/p>\n\n\n\n
Using ImageMagick, thumbnails of each pdf page are prepared, with a low but readable quality<\/p>\n\n\n\n
This result is stored in an archive directory structure, together with a JSON file containing the most important information. This little JSON file is created directly in the shell script. Low tech indeed.<\/p>\n\n\n\n
# Assemble the thumbnail subjason
<\/em>SEARCH=\"$OUTPUT_DIR${entry##*\/}\"
for JPGFILE in $SEARCH*.jpg; do
THUMBNAILS=\"$THUMBNAILS{\\\"imgname\\\" : \\\"${JPGFILE##*\/}\\\",\\\"imdirectory\\\" : \\\"\/$OUTPUT_DIR\\\"},\"
done
## strip the last \",\"
<\/em>THUMBNAILS=${THUMBNAILS::${#THUMBNAILS}-1}
JSON=\"{
\\\"document\\\" : {
\\\"name\\\" : \\\"$NAME\\\",
\\\"directoy\\\" : \\\"$OUTPUT_DIR\\\",
\\\"text\\\" : \\\"$PDFTXT\\\",
\\\"timestamp\\\" : \\\"$YEAR-$MONTH-$DAY-$HOUR-$MINUTE\\\",
\\\"origin\\\" : \\\"SCAN\\\",
\\\"thumbnails\\\" : [
$THUMBNAILS
],
\\\"tags\\\" : [
{
\\\"tagname\\\" : \\\"SCANNED\\\"
}
]
}
}\"
echo \"$JSON\" > \"$OUTPUT_DIR${entry##*\/}.json\"<\/pre>\n\n\n\nA .1-file is written in the import directory to mark a pdf as processed. I am planning to use the JSON file later as a document in Elastic Search.<\/p>\n","protected":false},"excerpt":{"rendered":"
A PDF arrives, a cron job checks once a minute if new files have arrived, it finds the file and starts a little processing chain. What I am using is OCRMyPDF to OCR the incoming file, it writes a new output file that has text underlay. With pdf2txt the text is exported and all blanks …<\/p>\n