Sponsored Links

High speed (multi GB) file storage

jon1

ULTIMATE Member
Was just wondering if anyone has any recommendations for high-speed UK (preferably London) based high-speed (ideally 10Gbps bandwidth) file storage servers

Have multiple team members handling 500GB-3TB files and we need to find a proper solution, fastest internet a team member has is 5Gbps, everyone else is either 1Gbps aside from the one very unlucky person with FTTC

Looking at some OVHCloud options but they take 1week+ to deliver, any recommendations?
 
Have multiple team members handling 500GB-3TB files and we need to find a proper solution, fastest internet a team member has is 5Gbps, everyone else is either 1Gbps aside from the one very unlucky person with FTTC
How much total storage would you need?
 
Sponsored Links
500GB - 3TB files and total of 10TB storage is somewhere between three to twenty files for the team.

If the person with 5Gbps connection has the space, wouldn't it be easier to put a fast NAS box there and share it with the other few members of the team?

It reads like there's a handful of users, rather than 100's.
 
500GB - 3TB files and total of 10TB storage is somewhere between three to twenty files for the team.

If the person with 5Gbps connection has the space, wouldn't it be easier to put a fast NAS box there and share it with the other few members of the team?

It reads like there's a handful of users, rather than 100's.
That is a good point. a good symmetric line (even a corporate account from an Altnet) might be perfect.

NVMe TrueNAS with a wireguard setup is easy. I run something similar on a 3gb symmetric.
 
Sponsored Links
Thanks for the answers guys, were looking at a NAS now

One last question, I seem to have incredibly slow upload speeds to a Texas, USA based production server with the following speeds:
7455cbd7-fa97-471f-b320-d774a00e01f7.png


I have a 1Gbps symmetric line, but my upload speed peaks at 150mbps
A team member in London with Community Fibre gets around 4x what I do to the server

Is this just simply because it's so far and theres nothing I can do to improve it?

Edit: I noticed that if I upload 2 files at the same time, they both reach 4.9 MiB/s and no further, 4.9 MiB/s is the max for a single file on its own

Edit 2: The issue goes away on a VPN
 
Edit: I noticed that if I upload 2 files at the same time, they both reach 4.9 MiB/s and no further, 4.9 MiB/s is the max for a single file on its own

TCP adjusts its speed in response to packet loss, which is part of its congestion control algorithm - but packet loss which is *not* due to congestion affects it in the same way. The maximum speed an individual stream can reach is given by the formula in the link above, and this effect gets larger with larger round-trip time (hence why you see this to USA).

But if you run multiple TCP streams to the same destination, each TCP stream gets the same limit separately. This is why multi-threaded speedtests can give higher performance than single-threaded ones.

If you can use a file transfer protocol which opens multiple connections (even Bittorrent or similar) you should be able to improve matters. Ideally you'd want to know where this source of low-level packet loss is, but it could be anywhere, including at the remote site itself.
 
Agree, been there, got that t-shirt in the past. I remember having to play around with a 1gig circuit between London and up north, RTT of around 10ms, and this silly "high performance" single threaded application would top out at around 200Mbits with the "network" being blamed by all stakeholders. With some TCP Window size adjustments got the performance closer to 800Mbits if my memory is not failing me.

RTT kills performance for any one TCP stream.
 
Thanks for the answers guys, were looking at a NAS now

One last question, I seem to have incredibly slow upload speeds to a Texas, USA based production server with the following speeds:
7455cbd7-fa97-471f-b320-d774a00e01f7.png


I have a 1Gbps symmetric line, but my upload speed peaks at 150mbps
A team member in London with Community Fibre gets around 4x what I do to the server

Is this just simply because it's so far and theres nothing I can do to improve it?

Edit: I noticed that if I upload 2 files at the same time, they both reach 4.9 MiB/s and no further, 4.9 MiB/s is the max for a single file on its own

Edit 2: The issue goes away on a VPN
build your own, some sort of 9th gen intel or newer, would easily hold out better than all the purpose build "NAS" noide 304 is the perfect case too.
 
Sponsored Links
Top
Cheap BIG ISPs for 100Mbps+
Community Fibre UK ISP Logo
150Mbps
Gift: None
Virgin Media UK ISP Logo
Virgin Media £22.99
132Mbps
Gift: None
Vodafone UK ISP Logo
Vodafone £24.00 - 26.00
150Mbps
Gift: None
NOW UK ISP Logo
NOW £24.00
100Mbps
Gift: None
Plusnet UK ISP Logo
Plusnet £25.99
145Mbps
Gift: £50 Reward Card
Large Availability | View All
Cheapest ISPs for 100Mbps+
Gigaclear UK ISP Logo
Gigaclear £17.00
200Mbps
Gift: None
Community Fibre UK ISP Logo
150Mbps
Gift: None
Virgin Media UK ISP Logo
Virgin Media £22.99
132Mbps
Gift: None
Hey! Broadband UK ISP Logo
150Mbps
Gift: None
Youfibre UK ISP Logo
Youfibre £23.99
150Mbps
Gift: None
Large Availability | View All
Sponsored Links
The Top 15 Category Tags
  1. FTTP (6026)
  2. BT (3639)
  3. Politics (2721)
  4. Business (2439)
  5. Openreach (2405)
  6. Building Digital UK (2330)
  7. Mobile Broadband (2146)
  8. FTTC (2083)
  9. Statistics (1901)
  10. 4G (1816)
  11. Virgin Media (1764)
  12. Ofcom Regulation (1582)
  13. Fibre Optic (1467)
  14. Wireless Internet (1462)
  15. 5G (1407)
Sponsored

Copyright © 1999 to Present - ISPreview.co.uk - All Rights Reserved - Terms  ,  Privacy and Cookie Policy  ,  Links  ,  Website Rules