I haved same problem, solution for me was turning on hide other effect option in options, i just didnt saw all towers effect with this option on and it failed me or gived me message out of time.
Yeah it is really annoying. My advice is just memorize one color and restart over and over until you memorized the right one.
Came here exactly to figure out if there was a timelimit, guess there is. Ain't no way you can do this solo with how short it is, the UI takes too long to even close itself so the timer just runs out before you can hit all pillars, yet YT videos show people take their sweet time and get them all in time.
Pearl Abyss should fix this, cause there ain't no way to Solo this, ESPECIALLY not if you give a Solo option.....