Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Image derivative creation should not fail on ImageMagick warnings #107

Open
scossu opened this issue Aug 31, 2016 · 8 comments
Open

Image derivative creation should not fail on ImageMagick warnings #107

scossu opened this issue Aug 31, 2016 · 8 comments

Comments

@scossu
Copy link

scossu commented Aug 31, 2016

Some TIFF files contain metadata that causes ImageMagick to exit with non-zero status. This causes the derivative job to fail.

This is my output from Sufia:

`mogrify -resize 200x150> /tmp/mini_magick20160829-32286-19a488t.tiff\` failed with error: mogrify: /tmp/mini_magick20160829-32286-19a488t.tiff: unknown field with tag 37724 (0x935c) encountered. `TIFFReadDirectory' @ warning/tiff.c/TIFFWarnings/715. mogrify: /tmp/mini_magick20160829-32286-19a488t.tiff: Unknown tag 37724. `TIFFSetField' @ error/tiff.c/TIFFErrors/499.

    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/mini_magick-4.5.1/lib/mini_magick/shell.rb:18:in `run'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/mini_magick-4.5.1/lib/mini_magick/tool.rb:92:in `call'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/mini_magick-4.5.1/lib/mini_magick/tool.rb:40:in `new'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/mini_magick-4.5.1/lib/mini_magick/image.rb:504:in `mogrify'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/mini_magick-4.5.1/lib/mini_magick/image.rb:395:in `method_missing'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/hydra-derivatives-3.1.1/lib/hydra/derivatives/processors/image.rb:32:in `block in create_resized_image'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/hydra-derivatives-3.1.1/lib/hydra/derivatives/processors/image.rb:38:in `create_image'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/hydra-derivatives-3.1.1/lib/hydra/derivatives/processors/image.rb:31:in `create_resized_image'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/hydra-derivatives-3.1.1/lib/hydra/derivatives/processors/image.rb:25:in `process_without_timeout'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/hydra-derivatives-3.1.1/lib/hydra/derivatives/processors/image.rb:8:in `process'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/hydra-derivatives-3.1.1/lib/hydra/derivatives/runners/runner.rb:32:in `block (2 levels) in create'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/hydra-derivatives-3.1.1/lib/hydra/derivatives/runners/runner.rb:29:in `each'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/hydra-derivatives-3.1.1/lib/hydra/derivatives/runners/runner.rb:29:in `block in create'
    /data/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/curation_concerns-1.2.0/app/services/curation_concerns/local_file_service.rb:7:in `call'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/hydra-derivatives-3.1.1/lib/hydra/derivatives/runners/runner.rb:43:in `source_file'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/hydra-derivatives-3.1.1/lib/hydra/derivatives/runners/runner.rb:28:in `create'
    /data/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/curation_concerns-1.2.0/app/models/concerns/curation_concerns/file_set/derivatives.rb:40:in `create_derivatives'
    /data/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/curation_concerns-1.2.0/app/jobs/create_derivatives_job.rb:10:in `perform'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activejob-4.2.7/lib/active_job/execution.rb:32:in `block in perform_now'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:117:in `call'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:505:in `call'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:498:in `block (2 levels) in around'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:343:in `block (2 levels) in simple'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/i18n-0.7.0/lib/i18n.rb:257:in `with_locale'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activejob-4.2.7/lib/active_job/translation.rb:7:in `block (2 levels) in <module:Translation>'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:441:in `instance_exec'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:441:in `block in make_lambda'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:342:in `block in simple'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:497:in `block in around'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:505:in `call'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:498:in `block (2 levels) in around'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:343:in `block (2 levels) in simple'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activejob-4.2.7/lib/active_job/logging.rb:23:in `block (4 levels) in <module:Logging>'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications.rb:164:in `block in instrument'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications.rb:164:in `instrument'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activejob-4.2.7/lib/active_job/logging.rb:22:in `block (3 levels) in <module:Logging>'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activejob-4.2.7/lib/active_job/logging.rb:43:in `block in tag_logger'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/tagged_logging.rb:68:in `block in tagged'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/tagged_logging.rb:26:in `tagged'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/tagged_logging.rb:68:in `tagged'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activejob-4.2.7/lib/active_job/logging.rb:43:in `tag_logger'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activejob-4.2.7/lib/active_job/logging.rb:19:in `block (2 levels) in <module:Logging>'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:441:in `instance_exec'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:441:in `block in make_lambda'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:342:in `block in simple'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:497:in `block in around'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:505:in `call'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:778:in `_run_perform_callbacks'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:81:in `run_callbacks'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activejob-4.2.7/lib/active_job/execution.rb:31:in `perform_now'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activejob-4.2.7/lib/active_job/execution.rb:21:in `execute'
    /usr/local/hydra/lakeshore/shared/bundle/ruby/2.3.0/gems/activejob-4.2.7/lib/active_job/queue_adapters/resque_adapter.rb:46:in `perform'

The mogrify command actually creates the correct derivative.

As per IM documentation [1] missing tags like the ones in the example above produce a status code between 300 and 399. Codes below 400 are considered warnings that still produce a valid image. Therefore the function calling the IM wrapper should not throw an exception if the return code is less than 400.

[1] http://www.imagemagick.org/script/exception.php
@mjgiarlo
Copy link
Member

Ew. Can you attach or link to a fixture file we can use to reproduce this error?

Thanks, @scossu!

@scossu
Copy link
Author

scossu commented Aug 31, 2016

Let me browse through the files that I have the right to publish...

@mjgiarlo
Copy link
Member

Much obliged.

@narogers
Copy link
Contributor

narogers commented Sep 2, 2016 •
edited
Loading

I can confirm this happens with certain images I try to process as well. Here's a gist that demonstrates the output which triggers the failure. In my case it is tied to some tags that Photoshop sticks in the EXIF metadata which ImageMagick does not consider valid.

https://gist.github.com/narogers/318d7dcbd34e06ae69140dcfbd2963d4

If using mini_magick (https://rubygems.org/gems/mini_magick/versions/4.5.1) I have found a temporary workaround by setting whiny = false in the application configuration (https://github.com/minimagick/minimagick#troubleshooting).

@mjgiarlo
Copy link
Member

mjgiarlo commented Sep 2, 2016

@narogers do you have any fixture images you may be able to contribute? bonus points for smaller files. ;) Alternatively, are you inclined to put in a PR for this issue?

@scossu
Copy link
Author

scossu commented Sep 2, 2016

Sorry about the radio silence on my end, I have been testing. I notice inconsistent results on different IM version. Our production server has an older version of IM and returns an error, the newer one not. Also the error code is 1, which may be a real error that does not return a valid image.

This is a related discussion including a sample file: http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=16390

@scossu
Copy link
Author

scossu commented Sep 4, 2016

@narogers Your solution sounds more than temporary if the whiny = false option suppresses only warnings, not real errors. It is not clear from the documentation you link to, though.

I would be interested in testing this option. Does this configuration change need any code change or is it something that I can override in my Sufia configuration?

@narogers
Copy link
Contributor

narogers commented Nov 1, 2016

@scossu I just dropped it into one of my initializers along with the rest of my ImageMagick configuration settings. It should be a minimal change.

https://github.com/ClevelandArtGIT/cma-archives/blob/master/config/initializers/mini_magick.rb#L10

@crowesn crowesn mentioned this issue Aug 28, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
Samvera Component Maintenance
Status: Backlog
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mjgiarlo @narogers @scossu