How do scientists and physicists know how big the universe is?

They say that the universe is about 13.8 billion lightyears across and expanding, but how do they know how big it is? If it is just how far away the farthest star is or something like that, doesn't it make that number purely speculative?