Compute Worker- reorganizing and cleaning of compute worker (#2294)

* reorganzing and cleaning of compute worker

* minor updates

* some more cleaning

* More cleaning

* Minor syntax cleanup

* RUNNING status during ingestion

---------

Co-authored-by: didayolo <adrien.pavao@gmail.com>

Author: ihsaan-ullah