So for anyone finding this issue and are in the same boat as me I can tell you that disabling innodb_adaptive_hash_index is the solution. After disabling it my server has not fallen behind once and the issue is absolutely with the Create/Drop of INNODB tables.
Hopefully this will help someone else in the long run, was a pain in the a** to find