- Joined
- May 19, 2017
- Messages
- 1,829
Good evening,
A few years past, I looked into the impact of L2ARC on rsync performance with FreeNAS. Back then (i.e. FreeNAS 11.x and earlier), the L2ARC was not persistent and for my use case (metadata only) it usually took about three passes before the L2ARC got "hot" (as of TrueNAS 12, the L2ARC can be made persistent - see here).
A metadata-only L2ARC is very beneficial for directory traversals and like operations: completion times associated with rsync of unchanged directories improved by a factor of up to 12x from the first to the fourth run. For example:
Whether or not sVDEV is the right answer for your particular use case is a different question, however. Remember, the L2ARC is not essential to the operation of the NAS. It can fail and then the NAS will use pool data instead. For performance and stability, stick to the recommendations here re: having at least 32GB of RAM before adding a L2ARC to avoid pressuring the ARC RAM. Especially if you're running jails or VMs.
Also: If your sVDEV fails, the pool will go with it. So use a muliti-drive mirror to host the sVDEV. Ideally, use high-grade SSDs from the likes of Intel rated for multiple wipes per day... especially if your NAS is hosting databases with many small files that are written and read constantly.
Lastly, regardless of how many resources you throw at the NAS, offsite backups are essential if you value the data.
A few years past, I looked into the impact of L2ARC on rsync performance with FreeNAS. Back then (i.e. FreeNAS 11.x and earlier), the L2ARC was not persistent and for my use case (metadata only) it usually took about three passes before the L2ARC got "hot" (as of TrueNAS 12, the L2ARC can be made persistent - see here).
A metadata-only L2ARC is very beneficial for directory traversals and like operations: completion times associated with rsync of unchanged directories improved by a factor of up to 12x from the first to the fourth run. For example:
- iTunes Folder (1.36TB of music files, iOS backups) 4th Run: 5 minutes, 4 seconds
- Pictures Folder (1.74TB of family pics) 4th Run: 42 minutes, 37 seconds
- Time Machine Folder (1.16TB of sparse bundles) 4th Run: 3 minutes, 29 seconds
- iTunes Folder (now 1.1TB of music files since I removed iOS backups): 1 minute, 42 seconds
- Pictures Folder (now 2.3TB of pictures): 34 minutes, 34 seconds
Whether or not sVDEV is the right answer for your particular use case is a different question, however. Remember, the L2ARC is not essential to the operation of the NAS. It can fail and then the NAS will use pool data instead. For performance and stability, stick to the recommendations here re: having at least 32GB of RAM before adding a L2ARC to avoid pressuring the ARC RAM. Especially if you're running jails or VMs.
Also: If your sVDEV fails, the pool will go with it. So use a muliti-drive mirror to host the sVDEV. Ideally, use high-grade SSDs from the likes of Intel rated for multiple wipes per day... especially if your NAS is hosting databases with many small files that are written and read constantly.
Lastly, regardless of how many resources you throw at the NAS, offsite backups are essential if you value the data.
Last edited: