Cheap 10 GbE SFP+ switch

[danb35]

From:

comes a reference to a small 10 GbE switch from Mikrotik--4x SFP+, 1x GbE, managed, for ~US$150. The link in the description didn't work for me, but it seems to be here:
https://www.neobits.com/mikrotik_crs305_1g_4s_in_cloud_router_switch_305_p13384368.html

 

[Chris Moore]

@danb35 Great item. Thanks for the info.

 

[Constantin]

Mikrotik has been a leader in bringing low-power but effective switches to market. I have a CSS326-24G-2S+RM (24-Gbit + 2x SFP+) unit at home that draws 9W - no Fans. Some of the online tests have questioned the throughput capacity of the CSS326-24G-2S+RM, especially if you're using it as a Layer 3 router. However, for my use case, (Layer 2 only, no routing, few simultaneous users), the thing has been impossible to saturate. Similarly, I expect the above CRS305 to be perfect for the "high-use" lanes in a home or office. Especially now that more and more WiFi AP's are potentially to make use of NBase-T connections above 1Gbit/s.

That said, the low price-point at Mikrotik comes with a caveat: You're pretty much on your own to set the thing up (they presume you're a network engineer) and technical support is somewhat non-existent. The Forums help make up for that... but not entirely. For example, I installed two CSS326-24G-2S+RMs at a pre-school and had one crash every two days like clockwork. No help from the OEM. Turns out it was likely a wiring issue - disconnected all cables that did not have a specific destination (bear with me, it's a pre-school without a cable plan) and the previously-unstable switch has been stable since.

Also wish that Mikrotik would enable their hardware to auto-update itself. Given the sheer number of exploits that their hardware has been subject to, it would be great if the gear could download updates at their leisure and then execute the update at a given time.

 

[Arwen]

That's a great piece of hardware! It's a switch, it's a router, it's either depending on your use.

Now all we need are some compatible & affordable SFP+ to 10GBase-T modules. These are a bit high in price, short in range, and I have no idea if they would work on anything but Cisco;
Amazon SFP+10GBASE-T Transceiver Copper RJ45 Module Compatible Cisco SFP-10G-T-S, Reach 30m

Edit: Looks like MikroTik has there own SFP+10GBase-T modules, though they say they need active cooling;
MikroTik S+RJ10
The neat thing about this one, is 6 speed support, 10/100Mbps, 1/2.5/5/10Gbps and 200 meter maximum length!

 

[Elliot Dierksen]

Looks like MikroTik has there own SFP+10GBase-T modules, though they say they need active cooling;

Interesting. It seems only the TP device mentions additional cooling. Their SFP+ modules are cheap enough that I would start there. Some switches are more picky than others.

 

[Constantin]

I’d use as many DACs as possible, esp. if the distances are short.

I just purchased a nbase-t thunderbolt 3 dock from akitio and the nbase-t SFP+ module from mikrotik to see how well it will all work on the cat5e in my home. Some folk have reported 10Gbit/s speeds over Cat5e with this gear. But realistically, even 5Gbit/s is likely faster than my NAS can handle.

 

[danb35]

If you're looking for fiber SFP+ optics, it looks like fs.com can supply compatible units for $16 each, though it appears you need to "customize" them. They have copper SFP+ units too, but at a much higher cost.

 

[Constantin]

That’s the irony, isn’t it? The optics are cheap, yet most vendors of a stand-alone thunderbolt 3 solution go the copper route and don’t offer a SFP+ option unless you want to go sky high re pricing.

It was cheaper for me to buy a external PCIe Thunderbolt expander box and put a eBay Myricom SFP+ adapter in it than to buy a standalone SFP+ to Thunderbolt 3 adapter. Nuts.

There is hope that as Apple and other vendors start mass-producing NBase-T capable hardware ($100 Mac mini option, for example) that the market will follow.

 

[danb35]

Looks like it's about $10 cheaper from Baltic Networks, if you can wait a few weeks:
https://www.balticnetworks.com/mikrotik-4-port-sfp-802-3at-af-switch-l5.html

 

[Arwen]

Interesting. It seems only the TP device mentions additional cooling.
...

Correct, at least that is the way I read it. The manufacturer wants them installed on switches with active cooling. Or, switches that are passively cooled, but have external cooling.

Originally there were no plans by anyone to release SFP+10GBase-T modules, as they drew too much power, (compared to optical). But, reducing the supported cable length, and or using lower power chips seem to have made them both feasable and a good option.

 

[danb35]

I went ahead and ordered one from Baltic Networks--I can wait a few weeks to save a few bucks. I'm not worried about copper SFP+ modules; I'll be using either fiber or DACs. I figure this will give me a few more 10G ports without having to buy another 52-port switch--even if it doesn't really save me any money compared to the Aruba switch @Chris Moore has posted a bit...

 

[Chris Moore]

I figure this will give me a few more 10G ports without having to buy another 52-port switch--even if it doesn't really save me any money compared to the Aruba switch

I had actually considered a second switch, and I agree that this would be a better solution. I might order one myself after Jan 1st.

 

[cobrakiller58]

I'm really curious if bell's GPON sfp will work in this since their modem only has 1000baseT ports and the single sfp it somewhat limits the 1.5Gbps they offer.....

 

[Roman]

Excellent gear!

 

[averyfreeman]

From:

comes a reference to a small 10 GbE switch from Mikrotik--4x SFP+, 1x GbE, managed, for ~US$150. The link in the description didn't work for me, but it seems to be here:
https://www.neobits.com/mikrotik_crs305_1g_4s_in_cloud_router_switch_305_p13384368.html

I've been watching this device - $110 is for pre-order (cheapest I've seen) up to $170 when in stock

I recently just got a used switch I wanted to give honorable mention - the Dell Powerconnect 7048P (also 7024 - P denotes POE model which is optional)

48 1Gb POE ports 803.2at (30w/port) + up to 4 SFP+, CX-4 (screw-type) or 10Gbase-T RJ-45 connectors via 2 expanders (mix and match!).

~80w idle power consumption w/out anything plugged in. Not quiet, but not particularly loud, either. Comes stock with 2x 10000 rpm 54 dBa fans.

Firmware still being released - latest update was Dec 2018

iperf3 pegged at 9.4Gbps no range via 15 ft DAC between my two ESXi hosts (FreeBSD and OmniOS vms) - just got straight up to 9.4 and stayed there. 1500 mtu.

I've had a few 10Gb switches with all different types of cabling and I've never seen speeds that consistent (or high).

Got mine for $90 plus $70 for two SFP+ expanders ($160 total) shipped on ebay

Edit: PCIe cards are Supermicro 1x port 82599ES

 

[jgreco]

I recently just got a used switch I wanted to give honorable mention - the Dell Powerconnect 7048P (also 7024 - P denotes POE model which is optional)

The 7048 is also a layer 3 switch, which means that it does routing (and filtering) in addition to layer 2 switching, and that happens at wire speed as well.

 

[danb35]

The 7048 is also a layer 3 switch

...as is the Mikrotik. Supposedly.

 

[jgreco]

...as is the Mikrotik. Supposedly.

Okay, well, ah, but as I was explaining to a new network engineer just earlier today, no.

Most network engineers understand "layer 3 switch" to mean that it actually does the routing at switching speed.

Look at routers - real routers, not crap-arse NAT gateways often incorrectly CALLED routers ...

I have, under my desk, serving as a pedestal for my office laser printer, an Ascend GRF-400. This 1997 era technological miracle was one of the very first line speed hardware routers that used custom ASICs to route packets at wire speed. It was capable of routing 32 x 100Mbps connections, and through the miracle of distributed switching and route table lookups, could manage 150K routes and do them all at wire speed. This is a full hardware router, and wasn't perturbed by such little things as PPS count. The ASICs cost a fortune to design. It cost around $80,000 new. It handled about a third to a half of the Internet traffic of a good fraction of the state I live in for about two years...

By way of comparison, today, Ubiquiti offers the Ubiquiti EdgeRouter Inifinity, an 8 port 10G router. It has two 40G PHYs hooked up to a rather beefy CPU. The difference between these two units is night and day; the Infinity can be brought to its knees pretty easily because the CPU is fondling each packet to figure out what to do with it. Under normal use this does not happen, but Ubiquiti is relying on "normal use" and not a DDoS attack or something like that.

Now I mention this because this is fundamental. The Mikrotik is not a layer 3 switch by any reasonable definition. Most of Ubiquiti's router products are not, either. There are some specific exceptions, but basically if the CPU has to get involved in packet processing, it's not a layer 3 SWITCH. It's a layer 3 software router.

The Mikrotik appears to be able to be configured in order to function as a layer 2 switch, which means that there's dedicated forwarding silicon of some sort that can handle line rate forwarding under adverse conditions. That's not *terribly* hard to do as 10G switching silicon is more than a decade and a half old at this point. I am very much hoping this turns out to be the case, in which case the Mikrotik is a potentially very exciting device.

But when packets need to traverse the CPU, then that's not so good. We simply do not have inexpensive CPU's that are fast enough to deal with the required PPS loads for quantities of 10G ports.

So you can actually get layer 3 switches at non-hideous prices if you shop the used market. I picked up a pair of Dell 8024F's for $400/ea recently for a new data center deployment. These will do basic routing and switching and will do it at wire speed. But they're big and loud and power hungry, because they've got these massive ASIC's in them.

 

[Chris Moore]

Okay, well, ah,

Interesting. I was looking at other offerings from MikroTik and I was curious if you would share your take on this product:

CRS309-1G-8S+IN
https://www.balticnetworks.com/mikrotik-8x-10g-sfp-ports-poe-cloud-router-switch.html

 

[jgreco]

Interesting. I was looking at other offerings from MikroTik and I was curious if you would share your take on this product:

CRS309-1G-8S+IN
https://www.balticnetworks.com/mikrotik-8x-10g-sfp-ports-poe-cloud-router-switch.html

I asked Mikrotik earlier this month if they'd send me one to play with. They said yes, I sent them an address, and haven't heard anything more.

My *IMPRESSION* is that this (or the smaller 4 port one) have the potential to be the ideal home lab switch especially for ESXi and FreeNAS. I expect that it has almost certainly got to be fine when running under the switch firmware, but I did want to put it under some vlan, etc., strain before I put anything more than a mention about it out there.

This is actually a very good time to evaluate something like this because I've got a LOT of gear in the shop including some full enterprise layer 3 switches that are being prepped for a deployment later this year. So I have a lot of opportunity to sit there with a short stack of gear on the bench and beat on stuff.

I suspect it'll be nice under switchOS. What I would love to find is that it can also be made to work well under routerOS, because a lot of small home labs do not have the ability to be hosting lots of random bits of gear.