It's not just sensor size, it's all the tech of the sensor plus the processors. For example the new Sony A9 is markedly better (scores wise) in noise handling than the lower megapixel A7SII and Nikon DF. What's really odd is that the A7SII is actually worse than the A7S where normally you'd expect newer tech to be better.
However, overall I personally believe that tech is far more important now than pixel size. We're seeing 24 and 30+ MP sensors showing markedly better noise handling scores than lower mp cameras. If pixel size was the predominant factor I don't believe we'd see this. What does seem apparent though that overall sensor size is still a predominant factor also. Compare the D850 and D500 which both show the same pixel density, the score for the D850 is much better. Of course, not a true comparison as the D850 has a BSI sensor, but you see it across the board comparing APS-C vs FF sensor of similar aged tech.
That being said, how these scores translate into real world results can be misleading. For example, the D500 performs better than scores would suggest imo.