Another axis: a GPU wants you to load a large batch of work and then start it - you can't be bouncing between CPU and GPU work all the time, but you can mix SIMD and non-SIMD instructions freely.