Does the header encryption in RAR influence the speed of password recovery?

Yes, greatly.

If you're trying to attack very large archive without header encryption you can face problems from the CPU side -- archive decryption and unpacking done wholly on CPU and 4xGPUs can simply provide more data than CPU(s) can handle.

About header encryption — if when you're trying to open RAR archive you're getting "Enter the password" message immediatly — without seeing even contents of archive — then it means "header encryption" was used. And for password recovery process it's better because we don't need to unpack whole file, just check several bytes within decrypted header.

If you can see archive contents then we're need to unpack at least one file to test correctness of password.

If this is Solid archive first file will be chosen, if not solid — smallest one. If size of this selected file is bigger than several kilobytes you can face problems with CPU back-end.