Infos:
- Used Zammad version: 6.2.0-1705322325.8ff65214.bookworm
- Used Zammad installation type: Package
- Used Elasticsearch vesrion: 7.17.16
- Operating system: Debian 12
- Browser + version: N/A
Expected behavior:
- Search index rebuild is successful for Elasticsearch
Actual behavior:
- Search index rebuild errors out at certain tickets with:
ArgumentError: invalid byte sequence in UTF-8
Here’s what a full example of what an error looks like with ticket referenced:
Dropping indexes... done.
Deleting pipeline... done.
Creating indexes... done.
Creating pipeline... done.
Reloading data...
- Chat::Session...
done in 0 seconds.
- Cti::Log...
done in 0 seconds.
- Group...
done in 0 seconds.
- KnowledgeBase::Answer::Translation...
done in 0 seconds.
- KnowledgeBase::Category::Translation...
done in 0 seconds.
- KnowledgeBase::Translation...
done in 0 seconds.
- Organization...
done in 0 seconds.
- StatsStore...
done in 0 seconds.
- Ticket::Priority...
done in 0 seconds.
- Ticket::State...
done in 0 seconds.
- Ticket...
rake aborted!
Unable to send Ticket.find(240102).search_index_update_backend backend: #<ArgumentError: invalid byte sequence in UTF-8>
/opt/zammad/app/models/concerns/has_search_index_backend.rb:200:in `rescue in block in search_index_reload'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:194:in `block in search_index_reload'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/parallel-1.23.0/lib/parallel.rb:627:in `call_with_index'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/parallel-1.23.0/lib/parallel.rb:394:in `block in work_direct'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/parallel-1.23.0/lib/parallel.rb:637:in `with_instrumentation'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/parallel-1.23.0/lib/parallel.rb:393:in `work_direct'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/parallel-1.23.0/lib/parallel.rb:285:in `map'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:191:in `search_index_reload'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:42:in `block (5 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:41:in `block (4 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:39:in `each'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:39:in `block (3 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:60:in `block (3 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/opt/zammad/bin/bundle:121:in `load'
/opt/zammad/bin/bundle:121:in `<main>'
Caused by:
ArgumentError: invalid byte sequence in UTF-8
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8/lib/active_support/core_ext/object/blank.rb:127:in `match?'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8/lib/active_support/core_ext/object/blank.rb:127:in `blank?'
/opt/zammad/app/models/ticket/search_index.rb:86:in `search_index_attribute_lookup_file_oversized?'
/opt/zammad/app/models/ticket/search_index.rb:52:in `block (2 levels) in search_index_attribute_lookup'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/app/models/ticket/search_index.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `each'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:71:in `block in find_each'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:138:in `block in find_in_batches'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:245:in `block in in_batches'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `loop'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:229:in `in_batches'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:137:in `find_in_batches'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8/lib/active_record/relation/batches.rb:70:in `find_each'
/opt/zammad/app/models/ticket/search_index.rb:22:in `each'
/opt/zammad/app/models/ticket/search_index.rb:22:in `search_index_attribute_lookup'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:137:in `search_index_update_backend'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:195:in `block in search_index_reload'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/parallel-1.23.0/lib/parallel.rb:627:in `call_with_index'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/parallel-1.23.0/lib/parallel.rb:394:in `block in work_direct'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/parallel-1.23.0/lib/parallel.rb:637:in `with_instrumentation'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/parallel-1.23.0/lib/parallel.rb:393:in `work_direct'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/parallel-1.23.0/lib/parallel.rb:285:in `map'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:191:in `search_index_reload'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:42:in `block (5 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:41:in `block (4 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:39:in `each'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:39:in `block (3 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:60:in `block (3 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/opt/zammad/bin/bundle:121:in `load'
/opt/zammad/bin/bundle:121:in `<main>'
Tasks: TOP => zammad:searchindex:rebuild
(See full trace by running task with --trace)
Inspecting the ticket doesn’t reveal any unusual characters. I’ve even attempted to delete a ticket where the ticket was non-essential but it keeps occuring on multiple tickets so impossible to delete them all.
Steps to reproduce the behavior:
- Try to rebuild the search index using this command:
zammad run rake zammad:searchindex:rebuild
All was working fine in previous versions of Zammad but now can’t rebuild search index for Elasticsearch due to this error. Any help appreciated.