A digital system is a mathematical system in which numbers shift between discrete states instead of rotating continuously. For example, an abacus was a digital calculator because in it a bead had to be all the way up or all the way down; there was no meaning for a bead partially up or down. Electronic calculators and modern computers are also examples of digital systems, because they depend on logic gates which need to be either on or off - there are only two possible states.
Converting between analog and digital
Converting between analog and digital systems, such as recording sound on a computer, gives rise to two possible problems. First, data is rounded to the nearest discrete state. For example, suppose the analog signal has values 2.91, 2.99, and 3.04. If the digital system only has room to store integers, all three values will be stored as 3. A digital system with more possible states - for example, each tenth rather than simply each integer - can mitigate this problem, as 2.91 will round down to 2.9 instead of up to 3. However, it cannot completely solve this problem: 2.99 and 3.04 will both still round to 3. Modern digitalization systems attempt to make these errors imperceptible by the human ear.
The second problem is that data is only recorded at discrete times. We do not know what the signal - for example, the sound wave - is doing between the measurements. More than one possible wave can fit. This problem is called aliasing; it can also be mitigated by taking more frequent measurements but never totally solved.