Unity automatically imports assets and manages various kinds of additional data about them for you. Below is a description of how this process works.
When you place an Asset such as a texture in the Assets folder, Unity will first detect that a new file has been added (the editor frequently checks the contents of the Assets folder against the list of assets it already knows about). Once a unique ID value has been assigned to the asset to enable it to be accessed internally, it will be imported and processed. The asset that you actually see in the Project panel is the result of that processing and its data contents will typically be different to those of the original asset. For example, a texture may be present in the Assets folder as a PNG file but will be converted to an internal format after import and processing.
Using an internal format for assets allows Unity to keep additional data known as metadata which enables the asset data to be handled in a much more flexible way. For example, the Photoshop file format is convenient to work with, but you wouldn’t expect it to support game engine features such as mip maps. Unity’s internal format, however, can add extra functionality like this to any asset type. All metadata for assets is stored in the Library folder. As a user, you should never have to alter the Library folder manually and attempting to do so may affect the functioning of the project in the Unity editor. However, it is always safe to delete the Library folder (while the project is not open in Unity) as all its data is generated from what is stored in the Assets and ProjectSettings folders. This also means that the Library folder should not be included in version control.
Unity allows you to create folders in the Project view to help you organize assets, and those folders will be mirrored in the actual filesystem. To move assets around or rename them, it is best to do so within the Project view itself. If you use the filesystem/desktop to move/rename the files, make sure to also move/rename the respective .meta files which are found right next to the files they pertain to (if you have “Hidden Meta Files” enabled, you may have to show hidden files in Explorer/Finder). Otherwise Unity-specific data for the assets will be lost which may lead to import settings and/or links between assets not being preserved.
When backing up a project, you should always back up the main Unity project folder, containing both the Assets and ProjectSettings folders. All the information in these folders is crucial to the way Unity works. You can ignore the Library and Temp folders for backup purposes.
Note: Projects created in Unity 4.2 and earlier may not have .meta files if not explicitly enabled. Deleting the Library folder in these projects will lead to data loss and permanent project corruption.