CPU, memory, but even graphics cards etc are nowadays integrated circuits (ICs, chips). ICs contain mainly (millions of) transistors. Transistors have three external connections and are used to amplify signals or as a switch. Several transistors combined yield a logical (AND,OR,NOT,...) gate. Several gates combine into a logic circuit. ICs also contain resistors and capacitors. ICs are made of one block of semiconductor material---silicon or gallium arsenide. The transistors are ... The design (layout) of the chip is very important, and a lot of research is done on that.
Between 1947 and 1958, before the invention of ICs, the transistors were just glued together. Before the invention of transistors in 1947, vacuum tubes (CRTs) were used instead of transistors.