Not sure I see the logic behind the encode -> decode -> display workflow as opposed to just display, neither can I see why you can't just use the FFmpeg wrapper end-to-end as libavcodec already wraps x264 (also, gotta say, having worked with the x264-rs repo you linked a bit, the code quality leaves something to be desired), but yes you can feed raw NAL units independent of an input file.
What you're looking for is [`Packet::copy`](https://docs.rs/ffmpeg-next/4.3.6/ffmpeg_next/codec/packet/packet/struct.Packet.html#method.copy) which bridges the gap by wrapping your NAL byte buffer into a packet libavcodec can work with. Then you just feed the packets into a decoder as usual.
I adapted the encode example you linked to into something that decodes the encoded NAL units and dumps the decoded frames as images (of course you can do anything else with the decoded frames, including displaying them): https://gist.github.com/zmwangx/0efa56bba9daef3d460b9ccf3a4832df
As for latency, you'll just have to write benchmarks.
Oh, I took a look at https://docs.microsoft.com/en-us/cpp/c-runtime-library/errno-constants?view=vs-2019, saw that POSIX errnos are supported for compatibility, so didn't check if all are actually available.
Since those three are reserved in POSIX, I'm simply dropping those. I took the liberty to modify the commit that way. Merged in 6a7a69e1e7d6d86a09f6360bd973ff249df46f89 (authorship still yours).