10 Gbps from the pole, how do I effectively do this?

S

sfsuphysics

[H]F Junkie
Joined
Jan 14, 2007
Messages
15,734
Ok just got pushed in a direction for 10 Gbps fiber, and I'm a little bit at a loss on how to wire this up. Now I know I won't be able to utilize all of it, not through wired connections in the home, not through wireless, not through every member of the family using it all simultanously (all 3 of us, one of us is 5years old), and I don't have any work related stuff that would in any way need even 1/10th this speed, but I am wondering if I can in any way set things up for the future now.

Now why 10Gbps? Why not just go with 1Gbps fiber? Well I can get 1Gbps fiber too, but it's the same cost, $40/month, they basically supply the line and don't have different pricing base on speed the line can handle (unlike literally every major ISP). So they're still wiring the neighborhood, so I have some time to plan things out, supposedly they will wire fiber and give an ONT so things are good to go from the pole to just inside the house.

So my existing router still works, and honestly nothing that connects to it wireless or wired (other than my computer) can get close to even 1 Gbps speed, and I doubt other than downloading games from very well established companies during off peak times would I even get anywhere over 1Gbps. And ultimately, at least in the near term, I'm probably going to treat it as if it's a 1 Gbps line since all the existing hardware is that. But if I wanted to bring the full power of the darkside (10Gbps line) to my rig, what's the cheapest way to go about that? Could I plug directly into the computer and then have a line from the computer that goes to the router for everything else ? I realize I'd need a 10GbE card or something first. Also could that work if the main rig isn't on all the time? Or would I want to go through a 10 GbE switch and just end up spending a bunch of money for that? I just don't want to go all LTT here because unlike him resources are limited here.

Sorry for the n00bish questions, I just have never had an option where the incoming line was so fast that it ended up being faster than what my existing hardware could handle, my jump from 6Mbps DSL to 100+ Mbps was well after Gigabit ports were common place, and the closest thing to an upgrade I had to do was upgrade from I think DOCSIS 2.0 to 3.0. Just wondering if I should wait out the storm until eventually 10 GbE becomes so common place that it'll be "cheap"?
 
First off, jealous. I can get 1G fiber for $70ish/mo, but only if I pay a car for installation, so that's not happening here. :(

If it were me, I've got a desktop 'server' as my internet router, so get a 10G card for that and maybe set up LACP to the nearby switch and between that switch and other switch so the server can do 10G and everyone else can share 2G (limited to 1G max for each).

Mikrotik has some 10G gear that approaches affordable, but I don't know if it does NAT.
 
What is your budget range?

Putting your desktop directly on the public internet and not behind a firewall just to get raw 10gbps is not a good idea.
 
Get a 10Gb sff machine, lock down the bios, and install a firewall OS on it...or get a 10Gb firewall. Connect the firewall to a 10Gb switch.
 
ComputerBox34 said:
What is your budget range?

Putting your desktop directly on the public internet and not behind a firewall just to get raw 10gbps is not a good idea.
Click to expand...
Don't really have a budget range, I'd rather not spend much though maybe price of a typical router price so ~$200 range which is why I figure plan B which is why I figure just deal with it as a 1Gbps line plugged into the current router = free until the price of said 10GbE equipment comes down to Earth, hell everything I see listed as "10GbE routers" only have 2.5 Gb LAN input ports, so I guess they're counting all the wireless traffic you could have.
And yeah, I thought about that right after I said it. Windows Defender probably ain't going to cut it :D
 
Last edited:
I would recommend investing in a new router that can handle 10G, such as this one ($700) as an example:
https://shop.netgate.com/products/6100-base-pfsense
And before you ask, yes, the 4 WAN ports are just generic interfaces that can each be configured separately as VLAN, WAN, LAN2, etc. This kind of device future-proofs you for dual 1G or dual 10G WAN for the (near) future.

Alternatively, you're back to building a dedicated PC with a 10G PCIe network card and multiple 1G PCIe cards to act as a router/firewall.

A third but not-recommended option, you could buy a managed switch with a 10G uplink (fiber) interface and properly connect it to your existing firewall/router... Misconfiguration here can be very bad (security-wise) or frustrating, so I don't recommend this option for the faint of heart.
 
May I ask where in the world you are and maybe the ISP just so I can look and be jealous? I have 1Gbps but hot darn!

First let me say that in my limited work, Fiber is not like ethernet. My Fiber is different that what I'm use to. It's an LC connector but single, I'm use to dual. Also the transceivers I work with are type and speed specific. Single mode, Multi Mode, 1Gbps, 10Gpbs (haven't gotten anything above that yet) So you'll want to know the specifics of the connection. It may not matter as you'll likely be plugging a transceiver to an SPF+ port and have some options but do understand that there are different types and even if you just want to run at 1, you'll probably still be required to connect at 10.

I like djstarfox's idea, I've never even heard of that company but it's a solution. Directly connecting a machine isn't recommended but it would work and would be one of the cheaper options. If you're going to buy cards though, maybe look into the 2.5Gbps wired stuff? It falls back to 1, 100, and 10 so there is some future proofing there.
 
bigstusexy said:
May I ask where in the world you are and maybe the ISP just so I can look and be jealous? I have 1Gbps but hot darn!

First let me say that in my limited work, Fiber is not like ethernet. My Fiber is different that what I'm use to. It's an LC connector but single, I'm use to dual. Also the transceivers I work with are type and speed specific. Single mode, Multi Mode, 1Gbps, 10Gpbs (haven't gotten anything above that yet) So you'll want to know the specifics of the connection. It may not matter as you'll likely be plugging a transceiver to an SPF+ port and have some options but do understand that there are different types and even if you just want to run at 1, you'll probably still be required to connect at 10.

I like djstarfox's idea, I've never even heard of that company but it's a solution. Directly connecting a machine isn't recommended but it would work and would be one of the cheaper options. If you're going to buy cards though, maybe look into the 2.5Gbps wired stuff? It falls back to 1, 100, and 10 so there is some future proofing there.
Click to expand...
Don't be jealous, no one is jealous of those who live in San Francisco (and that's without getting political at all)... ok maybe people can be jealous of all the food options, but that's another topic.

The company is Sonic.Net. I want to say they're a local company though, they do not have anywhere near the service areas as major ISPs though, they primarily are the SF Bay Area, but a quick google does show other cities. The "problem" with them, is that rather than just over extend and build out everywhere only to have it come bite them in the ass their business model has been a slow expansion, and it seems to work, which is why they can charge just $40/month. The thing is if you get their DSL service, it's $40/month, 1Gbps fiber $40/month, 10Gbps fiber $40/month. I seem to recall their CEO saying "bandwidth is cheap, it's the cost of running lines and maintaining them that really costs money" now how true that is I don't know, but I like that mentality, but it does explain their relatively small coverage area they simply can't afford to bring fiber to say a neighborhood if there's not many that want it because they're happy with what other option they have. They have been around a long time though, so it's not just a fly by night ISP that just popped up and will go bankrupt later. More at the end of this post with my story with them. But initially they were in the game of leasing lines from the phone company, still are to an extent, so that pricing will be comparable to the phone company costs, however in the neighborhood they're actually running their own fiber, so now we get the $40/month rate and we're happy... although I currently have Xfinity for $30/month for 400Mbps speeds, so who knows. competition is a funny thing how it works.

Bottom line though, it seems like it would be more expensive than I care to do with at the time to try and get 10G speeds, and considering there's really not much use in that much speed for me now, so if I were to go with them I'd just forcibly throttle myself to 1Gbps because that's what the equipment I have can do. There's no point in "future proofing" it seems, as it would cost a lot more now than later, and effectively have the same result I wouldn't really benefit now and it'd just cost more money now.
 
sfsuphysics said:
Don't be jealous, no one is jealous of those who live in San Francisco (and that's without getting political at all)... ok maybe people can be jealous of all the food options, but that's another topic.

The company is Sonic.Net. I want to say they're a local company though, they do not have anywhere near the service areas as major ISPs though, they primarily are the SF Bay Area, but a quick google does show other cities. The "problem" with them, is that rather than just over extend and build out everywhere only to have it come bite them in the ass their business model has been a slow expansion, and it seems to work, which is why they can charge just $40/month. The thing is if you get their DSL service, it's $40/month, 1Gbps fiber $40/month, 10Gbps fiber $40/month. I seem to recall their CEO saying "bandwidth is cheap, it's the cost of running lines and maintaining them that really costs money" now how true that is I don't know, but I like that mentality, but it does explain their relatively small coverage area they simply can't afford to bring fiber to say a neighborhood if there's not many that want it because they're happy with what other option they have. They have been around a long time though, so it's not just a fly by night ISP that just popped up and will go bankrupt later. More at the end of this post with my story with them. But initially they were in the game of leasing lines from the phone company, still are to an extent, so that pricing will be comparable to the phone company costs, however in the neighborhood they're actually running their own fiber, so now we get the $40/month rate and we're happy... although I currently have Xfinity for $30/month for 400Mbps speeds, so who knows. competition is a funny thing how it works.

Bottom line though, it seems like it would be more expensive than I care to do with at the time to try and get 10G speeds, and considering there's really not much use in that much speed for me now, so if I were to go with them I'd just forcibly throttle myself to 1Gbps because that's what the equipment I have can do. There's no point in "future proofing" it seems, as it would cost a lot more now than later, and effectively have the same result I wouldn't really benefit now and it'd just cost more money now.
Click to expand...
Yes, the food is really good (especially Indian--awe-than-tik). But yes, otherwise there are a lot of sucky stuff, especially in the city where every other street looks like something out of a national geographic of some third world nation. :eek:

Congrats on finding what sounds like an awesome isp. I wish I could have that type of a connection. I deal with packet lossy cable connections at every site I deal with. :mad:

So if I was in your shoes, I would get a used enterprise router good for 10G that has an SFP+ and go from there. You can run a dac to your computer from the router and at least have full 10G there. But it's not cheap as 10G isn't cheap in terms of a router. The cards and switches are reasonable if you know where to look and I'd do some searches and reading over at the servethehome forum as those guys will run stuff as fast as 100G in their labs and 10G is like 100Mbs for the rest of us. :D

I'd stay away from all the consumer stuff even if it says 10G--there's nothing but fluff and marketing in that stuff even if Asus and TP-Link make some decent products, but just not for the routing part of it.
 
SamirD said:
I'd stay away from all the consumer stuff even if it says 10G--there's nothing but fluff and marketing in that stuff even if Asus and TP-Link make some decent products, but just not for the routing part of it.
Click to expand...
Yeah I was finding that out in a hurry as I found a bunch of "10Gbps routers" and yeah they were a bit more pricey in the $500 range, they all seemed to be 10Gbps worth of wireless traffic they can handle with multiple channels but still were 1Gbps wired in connection.
 
sfsuphysics said:
Yeah I was finding that out in a hurry as I found a bunch of "10Gbps routers" and yeah they were a bit more pricey in the $500 range, they all seemed to be 10Gbps worth of wireless traffic they can handle with multiple channels but still were 1Gbps wired in connection.
Click to expand...
You need to be looking for devices capable of 10Gb/s WAN throughput and you're not going to find that in even a $500 consumer wifi router.

Based on what I'm reading, it seems its possible you didn't really know much about the level of hardware required to truly support a 10Gb connection.
 
Unfortunately not much you can do with $250.

https://mikrotik.com/product/CCR1036-8G-2Splus#fndtn-specifications - this is about $1000 and will get you 6.5 gbps in best case scenarios.

https://shop.netgate.com/products/6100-base-pfsense - $700 and will get you between 3 - 6 gbps. It can reach 10 but it would have to be under the most ideal of scenarios with little firewall rules.

Keep in mind you will also need a downstream switch with minimum 10gbps uplink to connect to the router/firewall that also supports mgig interfaces to connect to your hosts or additional SFP+ interface if you just want to connect a desktop.

All in all, the entire setup will probably run you about $2500 between router, switch, SFP+ PCI-E adapter for you desktop, and SFP's.
 
ComputerBox34 said:
Unfortunately not much you can do with $250.

https://mikrotik.com/product/CCR1036-8G-2Splus#fndtn-specifications - this is about $1000 and will get you 6.5 gbps in best case scenarios.

https://shop.netgate.com/products/6100-base-pfsense - $700 and will get you between 3 - 6 gbps. It can reach 10 but it would have to be under the most ideal of scenarios with little firewall rules.

Keep in mind you will also need a downstream switch with minimum 10gbps uplink to connect to the router/firewall that also supports mgig interfaces to connect to your hosts or additional SFP+ interface if you just want to connect a desktop.

All in all, the entire setup will probably run you about $2500 between router, switch, SFP+ PCI-E adapter for you desktop, and SFP's.
Click to expand...
I think your looking in the wrong places. SFP+ hardware is easily found used for reasonable prices. I see ~$20 on eBay for a pcie adapter and switches in the $100 range.
 
criccio said:
Based on what I'm reading, it seems its possible you didn't really know much about the level of hardware required to truly support a 10Gb connection.
Click to expand...
Oh definitely, I thought I made this abundantly clear. Like I said never had the option of this much bandwidth that far exceeded the current level of consumer hardware commonly available. When I had DSL 10/100 was everywhere, when I got 200+Mbps gigabit ports were common place as well.
 
Spitballing here. Could you go something like they did here? Their 'budget' setup was about $325 and was only able to hit 6GBs but you might be able to bring that down some more with some different used equipment and maybe a 2port 10GB to start since that is all you need for the basics. That would cut the 10GB card almost in half. Since you don't think you are going to be pushing the full bandwidth initially, you could start with a smaller build still capable of pushing multiple GBs to take advantage of it and build out as you go.

https://drakeor.com/2021/04/14/setting-up-pfsense-as-a-router/

as of a 2018 post
https://linustechtips.com/topic/954529-to-build-a-10g40g-fiber-pfsense-router/
"
pfSense is 'currently' software bound for filtering speed and I was unable to packet filter much higher than 7Gbit/s**, depending on hardware that is. So if you are wanting more than that, you need to wait until they have done optimisations and added additional offloading which I believe is due in the 3.0+ versioning (supposedly).

**This is in my testing using 2x 10G x540's with a Dual Xeon R720 with 2x 2695 v2's and all the hardware acceleration features enabled**

Personally I would advise you to look at vyOS rather than pfSense for this task as it will give native switching speeds.
"
 
sfsuphysics said:
Don't be jealous, no one is jealous of those who live in San Francisco (and that's without getting political at all)... ok maybe people can be jealous of all the food options, but that's another topic.

The company is Sonic.Net. I want to say they're a local company though, they do not have anywhere near the service areas as major ISPs though, they primarily are the SF Bay Area, but a quick google does show other cities. The "problem" with them, is that rather than just over extend and build out everywhere only to have it come bite them in the ass their business model has been a slow expansion, and it seems to work, which is why they can charge just $40/month. The thing is if you get their DSL service, it's $40/month, 1Gbps fiber $40/month, 10Gbps fiber $40/month. I seem to recall their CEO saying "bandwidth is cheap, it's the cost of running lines and maintaining them that really costs money" now how true that is I don't know, but I like that mentality, but it does explain their relatively small coverage area they simply can't afford to bring fiber to say a neighborhood if there's not many that want it because they're happy with what other option they have. They have been around a long time though, so it's not just a fly by night ISP that just popped up and will go bankrupt later. More at the end of this post with my story with them. But initially they were in the game of leasing lines from the phone company, still are to an extent, so that pricing will be comparable to the phone company costs, however in the neighborhood they're actually running their own fiber, so now we get the $40/month rate and we're happy... although I currently have Xfinity for $30/month for 400Mbps speeds, so who knows. competition is a funny thing how it works.

Bottom line though, it seems like it would be more expensive than I care to do with at the time to try and get 10G speeds, and considering there's really not much use in that much speed for me now, so if I were to go with them I'd just forcibly throttle myself to 1Gbps because that's what the equipment I have can do. There's no point in "future proofing" it seems, as it would cost a lot more now than later, and effectively have the same result I wouldn't really benefit now and it'd just cost more money now.
Click to expand...
Thanks for the reply and the other stuff, I really do appreciate it. I live near Chicago so we're not slouching on the food lol. Around where I live and also where I work, it's not green field so Comcast isn't thinking of residential fiber, we have their business fiber though. No idea why one of our buildings didn't get in that contract the second time, but they have business coax and it sucks. First DOCSIS is horribly Asymetrical, I think we're 300/30? So the VPN is much slower than we'd like (not going to get into the stuff MS has done that make it even worse) but AT&T is starting to build out fiber more and more, I am speculating it is because not only do they need to ditch DSL but because of lots of new runs needed for 5G mmw and firstNet. I was paying $45 for DSL when I got the chance to get fiber (I shouldn't have it) and now it's $60, but I think it goes to $80~$85 after the promo for 1Gbps. I know their history and the CEO with "riding my pipes." I'd love to get that one building on even home fiber but so far even the polls aren't wired up. I can go several blocks away and see it's there but nothing near that building :/ Oh, this building is smack in the middle of a residential area.

So congrats, have fun, be safe.
 
bigstusexy said:
Around where I live and also where I work, it's not green field so Comcast isn't thinking of residential fiber, we have their business fiber though. No idea why one of our buildings didn't get in that contract the second time, but they have business coax and it sucks. First DOCSIS is horribly Asymetrical, I think we're 300/30?
Click to expand...
You should start seeing that changing a lot , especially with Comcast, or any Cable provider that is rolling out DAA. They'll be able to start rolling out 1 G symmetric speeds (if they haven't already) soon, and are seriously building towards Symmetric 10G (well, Symmetric 10G in the cable world is 10 down and 6 up, frequency allotment with FDX or ESD I assume, but it will eventually get to a real 10G Sym) probably starting in the 2024 range. Might see faster rollouts as Flexible Mac or more RMD come into play, along with vCMTS/vCCAp architectures.


bigstusexy said:
AT&T is starting to build out fiber more and more, I am speculating it is because not only do they need to ditch DSL but because of lots of new runs needed for 5G mmw and firstNet.
Click to expand...
Definitely more for the 5G cell sites, as you'll need tons (In the few hundreds to 10's of thousands depending on power and wavelengths) to cover the same radius as a traditional macro cell. Firstnet tends not to be as bad.
 
bigstusexy said:
AT&T is starting to build out fiber more and more, I am speculating it is because not only do they need to ditch DSL but because of lots of new runs needed for 5G mmw and firstNet.
Click to expand...

I think there's a few things going on. Where the lines are overhead, they've figured out how to make the fiber runs relatively simple to do with passive splitters at each pole (or every few poles in lower density areas) and a factory made fiber bundle with a customized length drop for each pole, so they don't need to do any field splicing to setup the bundles. A lot of their copper bundles are old enough that it's getting hard to find good condition pairs or all the good condition pairs are in use or both. I've also heard there's something about copper lineworkers being union and employees and fiber lineworkers being contractors, also fiber lines may not have the same regulatory requirements as copper. So TLDR, need to replace the lines, figured out how to do it cheaply, and try to avoid rules and unions. Also, the first big round of AT&T fiber runs was announced and then happened shortly after Google Fiber announced their list of 20 or so cities they were totally going to work on and AT&T did all the cities in the Google list that were in AT&T territory. In San Jose, AT&T did it before Google could even get sites selected for their fiber distribution equipment. Google then decided not to run anymore fiber, shocking no one.
 
Balkroth said:
You should start seeing that changing a lot , especially with Comcast, or any Cable provider that is rolling out DAA. They'll be able to start rolling out 1 G symmetric speeds (if they haven't already) soon, and are seriously building towards Symmetric 10G (well, Symmetric 10G in the cable world is 10 down and 6 up, frequency allotment with FDX or ESD I assume, but it will eventually get to a real 10G Sym) probably starting in the 2024 range. Might see faster rollouts as Flexible Mac or more RMD come into play, along with vCMTS/vCCAp architectures.
Click to expand...
And still with 1 TB for a data cap? :D
 
That's the second part of one of the huge reasons I want to stay away from Comcast. I've got no data caps. Even with my frustrated issues, I can still push push 20GB to and from work in a few mintues. If I was at full speed, busy and not paying attention, I could easily hit that cap.

Second is that I can run *almost whatever I want. They block port 25 outbound but when I was on VDSL, you could still call one of their pay service and get it removed, might do it again, might use a free relay. Also their speeds aren't symmetrical, finding the upload is freaking painful. They aren't all doom and gloom, that's just my comparison.

Their enterprise fiber is a different animal. Only bad thing to say about it is their sales. They are vicious! We had them bid one year and chose to go with AT&T instead. They sued us. They were sure they had the better pricing, and they did, but ratings were not solely based on price.

P.S. That is not how we ended up using them. They only provide the WAN and not the internet. In my state, a company basically runs their own network for entities that qualify. They decided to use that network in conjunction with public ones and do a group buy. We get money for such things too and together that completely pays for the connection. They chose Comcast for the wan, but AT&T for the connection that'll feed the internet.

P.P.S. I came here to mostly say I don't know if info is leaking into Google, or I've just been looking at 10G stuff but I've been getting so many 10G stories in my Google news feed lol. One of them was for a pie like SBC with 2 10GbE interfaces, I might have saved it.
 
You must log in or register to reply here.
Back
Top