In a recent post about expanding a volume, we covered a slow but sure way of making that LUNs are labeled so that we know which LUN in our volume config corresponds to which LUN in our RAID config. Bringing them on one by one definitely works, but takes time and many MDC restarts. While I was working on that, my coworker found a better...
Read More
We just expanded a volume for a customer, adding 2 times their existing storage. Apple does have a short section on the process in the Xsan Management Guide for Mac v1.2, but there are several other things to consider when doing this. The first is sizing. When expanding a volume, you are adding storage pools. Ideally for performance all the storage pools should be the...
Read More
We have recently been upgrading our clients’ Xsans from macOS Monterey to macOS Ventura. We had been holding off since, in our testing over the last year, we have never had a successful in-place upgrade. When trying an in-place upgrade on an MDC, the install progresses until the first restart. At that point, the Mac boots to recovery with an error message: “Your computer started...
Read More
When working with an Xsan volume with Access Control Lists (ACLs) enabled, it is critical that users log into their Macs with the accounts that are in the relevant groups. We recently helped a customer migrate several computers from local accounts to mobile accounts while retaining the user’s home folder. We are starting with a Mac with a local administrator account and another local account...
Read More
We have mentioned that we have several environments with multiple Xsans and clients that are connected to both of them. In all cases there are 2 Xsans. One hosts the production volume with fast storage and many clients for video, audio, and graphics production. The other Xsan hosts a nearline volume for synced backups of production and often another volume for a file share. These...
Read More
We posted previously about making manual profile changes when changing an Xsan’s .auth_secret. After some more testing, we have confirmed that rebuilding Open Directory also allows us to cleanly change the .auth_secret stored in Open Directory, so a newly generated .mobileconfig file will connect clients without modification. To do this we will need to stop the volume, remove all clients, remove the backup mdc, and...
Read More
In a previous article we covered rebuilding your Xsan completely by saving the /Library/Preferences/Xsan directory and using that to populate rebuilt MDCs. We recently had a situation where a backup MDC would not rejoin the Open Directory as a replica after a software update. As this was supposed to be a quick, safe update we really didn’t want to have to erase and reinstall everything....
Read More
When adding clients to an Xsan, the normal process is to install an Xsan profile on the client which tells the client how to talk to the Open Directory. Then the Xsan configuration is pulled down from Open Directory and the client is able to connect. Occasionally we need to modify that profile. Here are 2 ways we have changed the profile in certain circumstances.
We have been seeing spotlight processes (mds, mds_stores, mdworker_shared, etc) using a lot of CPU on some of our MDCs. By itself this isn’t a problem, but we have been dealing with some thermal issues in a couple of small server rooms. While the benefits of spotlight are nice, it isn’t worth the extra heat in this case. We tried disabling spotlight with mdutil, but...
Read More
We recently had a situation where an Xsan was misbehaving. Both MDCs were regularly freezing up and becoming unresponsive. We found that the systems were regularly creating core dumps and filling the /cores directory until the drive filled up. We tried to diagnose the cause of this, but were not able to figure it out from the core dump files. So we erased and reinstalled...
Read More
I have been seeing some weird behavior with software updates on our macOS Monterey test Xsan. After running the software update, the Xsan volume does not mount. Trying to see the volume with xsanctl list results in xsanctl: unable to connect to xsand: No such file or directory. Having seeing this before I try to bootstrap the xsand launch daemon, which also fails. The error now is service is disabled. To resolve...
Read More
When supporting Xsans with a lot of data, we occasionally need to remove a lot of data. As the year rolled over, we have created new folders for the 2022 projects and it is time to remove some old data from the production volumes. After confirming the tape archive and nearline volumes have the data, we are clearing up some previous year projects to free...
Read More
I’m sure there are many ways to manage permissions on an Xsan, but we primarily see 2. The first is that anyone that can connect to the san can see everything on the san. Access is handled by keeping the san connected computers physically secure and not allowing network access to the san volume(s). Then permissions on the volume can be set so that...
Read More
We recently had a Mac that wouldn’t mount the Xsan volume when rebooting. We would receive the error xsanctl: unable to connect to xsand: No such file or directory when running xsanctl mount VolumeName. After confirming that the fibre HBA driver was loaded and the LUNs were available, we thought to check the issue that the error message actually described.
We ran into an interesting problem on some of our Xsan connected systems. We created mobile accounts, but some of them wouldn’t show up at the Mac’s Login Window. These Macs are bound to an LDAP server and the user’s UniqueID (UID) comes from the LDAP server. The UIDs are assigned based on user type, so while employees are small numbers, contractors have ended up...
Read More
We have seen some systems where our Xsan volume(s) do not mount consistently on restart. We have seen this occasionally on both macOS Catalina and Big Sur. After confirming that the fibre channel HBA driver is loaded, and seeing that manually running xsanctl mount <volumeName> works as expected, we can turn to launchd to make sure the volume mounts.
We have seen several Big Sur Macs at different locations not mount their Xsan volume after a restart or power on. After some investigation, we have found that the fibre HBA driver is not loaded. This has happened with ATTO ThunderLink FC adapters, ATTO Celerity PCI cards, and Promise SANLink FC adapters, so it doesn’t appear to just be one bad driver. The first...
Read More
In macOS Monterey the Server.app interface for setting up and managing Xsan is gone. Apple has produced an Xsan Management Guide (as of 2021-11-2 available on the Apple developer site). In this post I will walk through creating a new Xsan from scratch on macOS Monterey. Our test environment is made up of 2 Intel Mac Minis with ATTO Thunderlink fibre adapters connected to a...
Read More
With macOS Monterey, Apple has announced that the Server.app interface for Xsan is going away. One small benefit to this is that we don’t need to wait for the official release of Server.app v5.12 to upgrade our Xsan to Monterey. Both Big Sur and Monterey have the same Xsan versions (Server Revision 7.0.1 Build 589[96634]). This post will show the steps I used to upgrade...
Read More