Any Oracle ZFS "nerds" here that can help?

[TheUsD]

Fist and foremost, I under TrueNAS and Oracle ZFS are not the same thing. Just hoping some Oracle users hang out here...

A customer of mine gave me two FS1, two Server X4-2 and 3 DE2-24'. The deal was I could have them as long as I did a deband of the hard drives and show proof.
Was looking for a way to accomplish this so I could still utilize the drives but with no chance of recovering data. I have never delt with Oracle devices before; only Dell SCVs / EMCs and TrueNAS.

Tried googling how to do these but not 100% sure what to google.

I was going to do each drive independently, but it looks as these 2.5" SAS drives have a proprietary SAS connection. Can anyone identify what the name of the connector is?

Also, what OS would I be loading on these FS1's?

Thanks in advance!

 

[HoneyBadger]

two FS1, two Server X4-2 and 3 DE2-24

I imagine this is all a single "FS1" system in a tall rack, and you've got the Pilot management devices (the "FS1") the two Controllers (the "X4-2") and the DAE's (DE2-24)

The good news is that the Pilot and Controller are x86-64 systems, and you should (in theory) be able to boot the DBAN OS from removable media and just tell them to blank everything. The DAEs can be left attached to a single Controller system, or cabled into another server and blanked there. Providing hard proof to the client might require you to use a more "corporate" wiping solution, depending on their industry and regulatory requirements.

A good place to start would be the deploy/install guide for the FS1 system (PDF Warning)

https://docs.oracle.com/cd/E57980_15/E52540_02/E52540_02.pdf

Can anyone identify what the name of the connector is?

Going to need a picture here to determine but it could be the 2.5" FC format.

Also, what OS would I be loading on these FS1's?

Running the stock Oracle software is probably not possible due to licensing reasons. You could likely run TrueNAS on the Pilots and Controllers, but CORE doesn't have the access to the shared-storage HA feature, so you won't be able to just keep the cabling as-is.

@Arwen you had some Oracle experience if I recall correctly - want to see what this lucky individual got a hold of?

 

[TheUsD]

I imagine this is all a single "FS1" system in a tall rack, and you've got the Pilot management devices (the "FS1") the two Controllers (the "X4-2") and the DAE's (DE2-24)

The good news is that the Pilot and Controller are x86-64 systems, and you should (in theory) be able to boot the DBAN OS from removable media and just tell them to blank everything. The DAEs can be left attached to a single Controller system, or cabled into another server and blanked there. Providing hard proof to the client might require you to use a more "corporate" wiping solution, depending on their industry and regulatory requirements.

A good place to start would be the deploy/install guide for the FS1 system (PDF Warning)

https://docs.oracle.com/cd/E57980_15/E52540_02/E52540_02.pdf

Going to need a picture here to determine but it could be the 2.5" FC format.



Running the stock Oracle software is probably not possible due to licensing reasons. You could likely run TrueNAS on the Pilots and Controllers, but CORE doesn't have the access to the shared-storage HA feature, so you won't be able to just keep the cabling as-is.

@Arwen you had some Oracle experience if I recall correctly - want to see what this lucky individual got a hold of?

I appreciate the advice and help.

Both the Pilots and Controller do not contain any disks. Two of the DAE's have 24, 900GB while the 3rd has 7, 3.4TB drives which connects to the Controllers via HBA 12Gb. Because of that, this is why I was interested in taking them. I was thinking / hoping I could just throw some ESXi on either the Pilots or Controllers for a new vCenter cluster and have some direct attached storage (not knowing what they were until you specified). I guess I'll need to dig deeper but as it stands, I'm not sure what I could use these for.

 

[rvassar]

@Arwen you had some Oracle experience if I recall correctly - want to see what this lucky individual got a hold of?

I thought Arwen was ex-Sun, thought she's never replied to my PM. I did 15+ years at Sun, and a couple dozen months at "Orc-hell" in two stints. My first impulse is to roll it out to a empty parking lot, plug it in to the mains without proper fusing and throw holy water at it until the grid operator drops you or the local FD shows up... But I will confess to some opinion bias on this topic, and perhaps worry that the OP can't afford the BIOS license fee's to even power it on for such a show.

Biased sarcasm aside... The DAE's should be standard SAS or FC. Post pics, so we can help you ID them.

 

[Arwen]

Yes, Arwen is ex-Sun, (actually 4 times ex-Sun), from Broomfield, CO. Plus. still supports Sun servers at her day job .

@rvassar, Sorry I did not reply to your PM back then. It's possible that I was ill or very busy at the time.


The X4-2 servers seem pretty normal x64, with the caveat that the network management GUI for remote console is probably old Java, (given the age of the servers). The DE2-24 with 2.5" drives also seems normal. Both should work with many Linux, and Sun / Oracle guaranteed support with RHEL 6.4 for the X4-2 servers. Which I would guess, any later version should work too. These even have 100/1000/10000 network ports.

The controllers seem to be X3-2 servers, also pretty normal x64. And support RHEL 6.1, (and others like MS-Windows Server).

Your comment about proprietary SAS connector is likely not quite descriptive. SAS drives have the ability to use a second path to the disks, either for High Availability or performance. So SATA disks can plug into a SAS back plane, because SAS is backward compatible. But, SAS disks can't plug into a SATA back plane, since they won't work on a SATA host bus adapter. This is one reason why some vendors simply make single pathed SAS back planes that also support SATA drives. Works for either disk type, (without the second path for SAS).


There should be little reason a modern Linux distro can't be booted on the servers, and see all the disks.

As for wiping the disks, if your contact is good with DBAN, you might look at how you can get a screen shot, (or simple photo of the video monitor), after a disk is successfully wiped. If you show all 24 disks per tray wiped, (one photo or screen shot per disk), and those other disks, that might be good enough.


As a side note, I like Sun servers, both SPARC & x64. In some ways they are both less clumsy and more open that Dell servers.

 

[rvassar]

Your comment about proprietary SAS connector is likely not quite descriptive. SAS drives have the ability to use a second path to the disks, either for High Availability or performance. So SATA disks can plug into a SAS back plane, because SAS is backward compatible. But, SAS disks can't plug into a SATA back plane, since they won't work on a SATA host bus adapter. This is one reason why some vendors simply make single pathed SAS back planes that also support SATA drives. Works for either disk type, (without the second path for SAS).

To my knowledge there are almost no Enterprise SATA only backplanes. SAS and SATA are electrically compatible, until you hit the 1 meter distance mark. The drive plugs are identical, save the mechanical keying. Anything longer than 1m you need to use SAS, or an expander that can convert. Otherwise, think of them as simply two different languages, one Latin and the other Italian. It's that kind of close, but as you mention, SATA drops the multipath, expanded command set, etc...

 

[Arwen]

In standard PC desktop gear, they tend to use SATA only back planes. But, as you point out, that's not Enterprise SATA.

To a person used to SATA only, (no SAS disk / SSD experience), a SAS disk connector may look like a "proprietary SAS connection".

 

[TheUsD]

The FS1 and DE2-24

 

[rvassar]

Standard end of the road Sun x64 kit, with that ugly red badge on it.

Was expecting pics of the "proprietary SAS connector".

 

[TheUsD]

Standard end of the road Sun x64 kit, with that ugly red badge on it.

Was expecting pics of the "proprietary SAS connector".

haha sorry about that. I wasn't sure what you were looking for in specific.

There should be little reason a modern Linux distro can't be booted on the servers, and see all the disks.

Statement then questions on this...

A co-worker wantes the X4-2 servers (sure no problem). All I want is the FS1 / FS2 along with the DE2-24P's.

Questions:
1) Is there a specific OS I would need to load on the FS1 and FS2 to see the drives on the DE2-24P's?
1.a) Could I accomplish this on ESXi? I currently have ESXi 7.0u3 loaded up and I understand the CPU is not compatible and it appears the last version of ESXi compatible with the HBA cards was 5.5.... In the hardware section, it see the cards but It cannot see the containers nor find the disks in the contains.

2) Do I need the X4-2 servers to complete this job (with any or specific OS)?

3) If I do not need the X4-2 servers, how do I factory-reset the DE2-24Ps? I've looked online but all I found were these instructions:
To perform a factory reset:

• BUI: Click the Factory Reset button on the Maintenance > System screen.
• CLI: Enter the maintenance system context, then issue the factoryreset command.
• GRUB: In the GRUB menu, add -c to the line beginning with kernel.

But I do not see these options available in the ILOM or KVMS

My goal (as briefly mentioned in question1) was hoping to load ESXi on both the FS1 and FS2 and then direct attach them using the HBAs to see the drives. I am absolutely open to any other recommendations that would be a better idea.

Alternatively, I wish TrueNAS supported external HBAs to connect to these storage containers. From my understanding it does not.


I appreciate any suggests and support!

 

[HoneyBadger]

Alternatively, I wish TrueNAS supported external HBAs to connect to these storage containers. From my understanding it does not.

I'm not sure how you got this impression, because TrueNAS absolutely supports external disk shelves and HBAs.

Can you please post a picture of the rear of the DE2 disk shelves, and of a single disk removed from within them?

 

[TheUsD]

I'm not sure how you got this impression, because TrueNAS absolutely supports external disk shelves and HBAs.

Can you please post a picture of the rear of the DE2 disk shelves, and of a single disk removed from within them?

Yes, I stand corrected as I loaded TrueNAS on one of the FS1's and connected the HBAe to the storage container.

My verbiage was not correct. What I meant to say was going HBA external to ESXi so that I could present the storage as if it was local (Unless I am incorrect about that as well...) and avoid using iSCSi. Wish I could afford the licensing and get the FC support with the TrueNAS.

Per your request, @HoneyBadger :

 

[HoneyBadger]

Yes, I stand corrected as I loaded TrueNAS on one of the FS1's and connected the HBAe to the storage container.

My verbiage was not correct. What I meant to say was going HBA external to ESXi so that I could present the storage as if it was local (Unless I am incorrect about that as well...) and avoid using iSCSi. Wish I could afford the licensing and get the FC support with the TrueNAS.

Per your request, @HoneyBadger :
View attachment 59710
View attachment 59711

Those drives appear to use standard SAS connectors, and the back of the servers/DAEs seem to be using standard external SAS cables (some 6G, some 12G) so there shouldn't be anything preventing you from cabling them in as direct-attached shelves.

ESXi can also handle direct-attached storage, but you'll likely need to replace the external HBAs in those units with RAID cards if you want to format and access them directly from ESXi with a level of redundancy.

Assuming the controller units are on the top of the bottom photo, any 12G capable SAS HBA should be supported under ESXi 7.0 - not entirely sure why it wouldn't reveal existing physical disks even if they had an unrecognized partition table.

 

[TheUsD]

Oh wow!

Great information to know! Thanks, @HoneyBadger. This gives me a whole new way to think of configuring our storage containers.

Just to confirm, I cannot connect HBAe from a TrueNAS to HBAe on an ESXi host, right?

 

[HoneyBadger]

Just to confirm, I cannot connect HBAe from a TrueNAS to HBAe on an ESXi host, right?

I don't believe there's any support for the SAS "target mode" for LSI HBAs in TrueNAS, which would be required to enable that direct "back to back" link that you're after.

There's some (very) unsupported work in progress on Linux, I believe, but that's well into the "here be dragons" territory, as well as currently being limited to Marvell SAS controllers I believe.

But a point-to-point network connection could easily be done at 25/40Gbps. Having a read through this exploratory NVMe-oF thread might be of interest as well:

NVMe-oF support planned?

Are there plans to support this eventually? The Linux kernel ships with a NVMe-oF target driver, that seems to perform decently. It appears to be a bit more performant over TCP than regular iSCSI. On top of that, it'd also allow to enable RDMA-backed block I/O with Windows via the Starwind...

www.truenas.com