Building a Quick and Cheap Nas with TRUENAS.
Hello folks, for this month’s blog since I’m transitioning into school, I thought I’d take a break from studying to look at some projects in the backlog. We won’t be on security strictly speaking, but we’ll focus on system administration topics. I’ve been researching a couple xss areas in the meantime, but until that work is done I’ll be talking about the engineering projects that are actually coming back with good results. Ignite was fantastic and so was the NextJS conference (Plasmic especially), but lets get on with the blog.
So this week I’m going over my NAS, or network attached storage device. I’ve wrapped up much of my cloud computing work and found that many of the tasks that are handled there are also doable at home. Notably, you can take disk snapshots, run virtual machines, and store data under lock, key as well as SMB. When I was looking at cloud gaming some time ago, I took note that the games are running over the network on hardware that’s outsourced to the CSP infrastructure. With a NAS, the system you run process your graphics, instructions, ram and so on. This is far from a new concept, but always fun to see in action with non-gaming service like Azure Arc with Google Cloud integration. For my purposes, we have a simple NAS to build from scratch. Our goal is to play some games from Steam. Lets get started.
The discount behemoth — It’s alive!:
If you’re looking for a high end NAS device with blistering gigabit speeds and SMB multiplexing, look away now! Halloween may be over, but the PC frankensteining has only begun. For the fun and full learning experience I’ve went ahead and built my own with a low tier budget. The server we have is a 2011 Dell Optiplex 990 with a measly 4 GB of DDR3, a 4th gen i3 and no onboard hard drives out of the box. That was fine since I planned on bringing my own; I’ve provided two 1 TB surrogate hard drives which were gathering dust. These old reliable drives are capable of running at 5200 rpm. Where was the OS? On a USB2 flash drive. We’ll be running an internet favorite for the OS, TrueNAS, which is a freeBSD tool with over a decade of support and a huge open source community. On a fun note, I haven’t used freeBSD since the days I was tinkering with KDE in college, so it’s not a completely alien OS to me.
Onto the first setup quirk. Where was the redundancy for the OS? Nowhere, which became an issue with a hardware failure when I started out. I ended up doing an install to one thumb drive and later on breaking it while opening the box. I ordered up a new drive with a warranty and the same storage space which cost less than nothing if you can believe it. Everything that was built or provided had its own odd bugs and quirks, most of which I go over in the rest of the blog.
As a bonus for hardware, the PC came with a CD drive. A retro media junkie like myself can use this with CD-R’s and burning images to relive the glory days of 00’s music. While quaint, disks are on their way out. Still, it looks like Sony will support every storage medium in the solar system, from Tapes for archiving to Ultra Blu rays to handle high density data. Nostalgia aside, One less drive freed up some space for me to add more drives once I get a distribution source.
Hardly Driving me crazy:
One thing I will mention here is how weird the storage market is. On the one hand, you have reputable sellers for HDD’s, SSD’s and flash media storage. On the other, you have this odd scheme going on with websites like eBay, Wish and Amazon. Often pre-packaged and/or configured storage is coming tweaked and repurposed from unsold, obsolete inventory. Drives that are advertised with 1000 GB or more of storage have some values changed internally to appear legitimate. Unwitting consumers would buy these and never use them to the point of realizing they’ve filled up their TB drive after only a measly 32 or 64 GB’s. Storage media may have come a long way with Moore’s law of doubling semiconductor performance yearly, but these scams are just ridiculous.
For legitimate use cases, if you’re going to use adapters for SATA to micro SD, just be aware of what you’re purchasing first. They will usually work, but the fake ones disguised as USB sticks are out there. I will say that these scams are absolutely absurd, and most tech savvy folks know how to avoid them, but not necessarily report them. This is yet another example of marketplace fraud & abuse and definitely worthy of its own writeup.
Onto legit media, SATA is naively supported on these drives, as is USB. USB performance is limited to 2 transfer speeds, so running even the OS off here is ill-advised, especially without redundancy. SAS drives are typically cheaper and made for server based NAS setups. These will not work on SATA without conversion tools or their own PCIe cards. This is one storage expansion option to reduce cost. Another thing is buying refurnished SATA drives, which can come with their own plethora of issues, specifically detailed if you look into SMART.
Dealing with smart errors:
SMART is a reporting standard for hard drive operability. It has telemetry for ambient temperature, sector integrity, power consumption, power-on count, and much more. Having the right tool to view this information matters. trueNAS has one built in, and for windows I much enjoy the Active@ suite of disk management software to view this. KillDisk gives a nice clean readout of any disk, usually if you’re intending to wipe it before use.
For one of my HDD disks we encountered 3 SMART errors, Off-line Uncorrectable, Current Pending Sectors and Spin up Retries. Typically more errors adds to the risk calculation on whether or not the drive should be replaced. If I were running a production server, 3 would be the magic number and worth seriously considering a replacement. Since we’re on a budget and still learning, I’m looking at replacement options for the future.
Unknown disk in TrueNAS:
When I first set the disk up into a pool in TrueNas, I thought I encountered another problem. I saw the laptop hand-me-down drive was recognizable. The larger drive however reported UNKNOWN as its type. It turns out this might just be because the drive I have was ripped out of a USB housed storage without native indicators. For my purposes it works, but is another thing to seriously consider for the risk appetite.
SMB Configuration, Steaming: Setting up SMB in TrueNAS requires 1. configuring a pool. 2. setting up a folder share and selecting SMB as its method. 3. Configuring permissions.
The first two were easy enough, and TrueNAS has a preset to dodge most, but not all, of the security settings for the time being. I know I know, I’ll go back and set it up once I replace the faulty drive.
An interesting quirk I found was that Steam really likes local MAC or Mandatory Access Control. In order to work with the drive, I had to assign the “everyone” group read write and execute permissions in order to view and use it. Steam also doesn’t like when you add your NAS as a drive on the network after its booted. Epic Game Store doesn’t mind and lets you install any directory you point it to. Anyway, if you have an issue with Steam after getting your NAS up, reboot your system and it should be able to tell you a drive is mapped and ready to go as a steam library folder.
Switching with a router?! It’s more likely than you think:
Well our NAS was ready to go, what about the home network. Well we have a google home as our front end as well as a Mikrotik router set up as a bridge for the rest of the devices for the network. Bridging is as easy as using the command line or the GUI’s of your choosing. Once your connections are bridged and your router-turned-switch gets an address, we had TrueNAS, the home router and my main multi headed PC ready to communicate over the network. To link it all together I cut my own cable! This was an experience I hadn’t had since high school and I thoroughly enjoyed it. I realized that there were some nasty bends and cuts in some wires as I went along, so we adjusted, corrected for errors, and moved on.
Physical, the OSI layer everyone forgets:
The initial transfer speeds of my NAS over SMB were 10 MBPS write and 30 read. This was with a mirroring setup and I wasn’t sure if the data was getting to the funky unknown drive. I tore it down in favor of a Raid0 type of setup, but nope, speeds were still bad. What gives?
After a few googles I took a look at the negotiation speeds reported by Windows, Mikrotik, and TrueNAS. I noticed the Mikrotik switch was reporting 100 MBPS speeds between my NAS and the switch, as well as my base PC.
I thought this was odd, so I and moved some cables around. That factor showed me I had a bad cable that was only getting 100 MBPS. This was probably due to some weird bends and twists. Turns out it wasn’t just the cabling.
The root of the speed problem:
So the router uses a PoE injector to get working. Well it turns out it doesn’t support Cat 5E standards. Ouch. That was slowing both my home internet network down as well as the other devices. A google speedtest informed me the device couldn’t reach speeds above 100 MBPS for internet. This was ignoring the full 200 MBPS I subscribe to. This was also in spite of the fact that Mikrotik reported 1 GBPS negotiation speeds. Windows while reporting the ethernet device was set to 1 GBPS, showed me it was in fact not the case and only running at 100 MBPS with a quick Powershell command [1]. We opened another bridge on the Mikrotik, stopped using the LAN-to-PoE- relegated the faulty cable to power-only and got the rest of it fixed in a hurry.
The post-setup result:
SMB is good, steam likes both, we got 20–40 MBPS transfers with write. Striped RAID increases performance of the old drives to a speed needed to run some moderately disk-reliant games. I tested Apex Legends which ran slower than my 7200 RPM onboard hard drive, but surprisingly playable. This does affect gameplay somewhat but for free I can’t complain here. Much faster speeds are possible with iSCSI and that could make for another fun exploration.
That’s all for now, I’ll be back with more content later this month. Until next time. Stay safe.
