Hi again, @metalinux!
I realize this is not a direct fix or directly related to your specific issue, but ... there is a page related to use of SmartCards for US Gov that makes reference to MacOS (linux spinoff) on two aspects of the problem, namely:
I realize that might be a stretch ... but you never know where an answer will come from. 
There is also a section specifically dedicated to Yubico-PAM module, if you haven't seen that yet, and a section specifically on Yubico PAM Single-factor configuration guide which might have direct relevance to your problem.
I underline that I offer these, even though I don't understand the problem in depth 
, but based on an abstraction of the issue you are facing for your context.
I also forgot to mention Arch Linux's page on Yubikey. Their pages sometimes have hints/workarounds at the bleeding edge.