Assuming you are talking about the main Unraid array, then array does NOT need to be down while it is rebuilding a data drive or building a parity drive contents. It will only be down if you do the operation in Maintenance mode. If however you have the array available while either parity or a data drive is being built then any read/write operations to the array badly degrade each others performance while they are both running, but as long as you are aware of this and the amount of read/write to data drives will be minimal this is normally an acceptable compromise. Note these statements apply to the main Unraid array which is not limited to XFS as drives can also be in BTRFS and XFS and they can be mixed - it is just in the main Unraid array each drive is a free-standing file system.
There is an exception to this statement in what is called the Parity Swap procedure where you are trying to simultaneously upgrade a parity drive to a larger one, and then use the old parity drive to replace a (possible failed) drive in the main array. In this case the array is not available during the initial phase while the old parity drive content are copied to the new larger parity drive, but it IS available at the second phase where the array drive that is being replaced is having it contents rebuilt.