All technology, whatever it is, starts big and complex and expensive.
As it becomes easier and cheaper to build, and the people bulding it learn more, it becomes smaller, less complex and cheaper.
I worked for IBM 35 years ago, and computers then filled a room, and cost the earth. They were probably less powerful than the average home calculator is nowadays.
If they had hard disks (some did not) they were the size of a small fridge and held less data than most memory sticks now.
So you may well ask, why did they not build computers as small then as they are today.
After all the technology for building say a hard disk is pretty much the same today as it was then. But then they were as big as a fridge, now they are the size of a bar of chocolate.
The first mobile phones were the size of brick, the first playstation 2 was about four times the size of the smaller playstation 2.
All technology has to start somewhere.
Memory stick techncology was "new" when the first stick came out (I think the first I had was 16Mb believe it or not) but over time they have learned to squeeze more and more data into the same space.
We will probably look back and laugh at some of the things we think are pretty cool today.
Your grandchildren will say "You mean you had to TYPE to get things into your computer"