ES full reindex with slow-down
scanDatabase can sometimes be waaaaaay faster than our ES queue processor and ES service itself.
Update queue size: 1559000: 1.5M elements in the reindex queue and that indicates that the indexing is falling behind of scanning.
The queue size should not be larger than 10,000 entries at any moment.
The queue grows that large because of
- Slow indexing
- Fast scheduling
Where is the bottleneck?