Some materials are neither simple liquids nor simple solids, and thus they do not fall clearly under the classical scheme of materials classification. These fluids are called "complex" and they possess mechanical properties that are intermediate between ordinary liquids and ordinary solids.
Perhaps the most important distinction between classical solids and classical liquids is that the liquids quickly shape themselves to the container in which they reside, while solids maintain their shape indefinitely. Many complex fluids are intermediate between solid and liquid in that while they maintain their shape for a time, they eventually flow. They are "solids" at short times and "liquids" at longer times. Polymers (like plastic materials) are examples of such complex fluids, or also many food products such as peanut butter.
There are also complex fluids that change from solid-like to liquid-like, or vice versa, when subjected to some modest pressure. Gels are example of such liquids. Some fluids change to solids when an electric or magnetic field is applied. A classical liquid or solid does not change character in response to a weak electric and magnetic field.
There are many other physical properties that may be complicated, such as materials which form a gel while you wait (JelloTM), or glasses. So be careful when you call something a liquid or a solid. It might be neither!