Hewlett Packard Enterprise (HPE) supercomputer to tell Singapore that it's hot, humid, probably going to rain

[Hamartia Antidote]

Singapore deploys HPE supercomputer for weather forecasting

Cray EX beast deployed to improve weather forecasting in country that has basically one season

www.theregister.com

Cray EX beast deployed to improve weather forecasting in country that has basically one season

HPE has booked another supercomputer win, this time providing compute power for the Meteorological Service Singapore (MSS) to deliver improved weather forecasting and tropical climate research for Singapore and Southeast Asia.

The new supercomputer is another based on the Cray EX architecture [PDF] using AMD Epyc processors, like the Frontier exascale supercomputer in the US, currently ranked as the world's fastest.

According to HPE, the new system delivers nearly twice as much performance as its predecessor at the MSS, with peak performance of 401.4 teraflops. It will help increase forecast accuracy by enabling improved numerical model configurations and analysis of ground and space-based weather observations with the agency's SINGV numerical weather prediction system. At least that's the intention.

SINGV was developed by the Centre for Climate Research Singapore (CCRS) at the MSS, in conjunction with the UK Met Office and others, and is configured specifically for weather forecasts and climate applications in Singapore and the surrounding region.

HPE claims its new supercomputer has the performance required for enhanced forecast post-processing algorithms that use machine learning techniques to improve the quality of forecasts. The computational power also allows higher resolution models to deliver improved weather and climate data.

One such example is a coupled ocean-atmosphere-land-wave modeling system (cSINGV). This was developed to better understand the strong feedback between the atmosphere, land and ocean, which have a significant impact on the weather and climate over Southeast Asia, HPE says.

CCRS Director Professor Dale Barker said the new supercomputer was designed in collaboration with the agency to provide a faster system with next-generation technologies to advance modeling and simulation tools.

"Our scientists and software engineers are committed to developing advanced modeling systems and examining complex data to provide timely weather forecasts for our nation, which due to the island's unique geological positioning, often experiences various weather processes on a daily basis," he said.

The new MSS system is rather more modest than the Frontier exascale supercomputer, comprising 196 of AMD's third-generation Epyc processors, but interconnected using the HPE Slingshot high-performance network fabric and Cray ClusterStor E1000 parallel storage, in line with other supercomputers based on the Cray EX design.

Oddly, neither HPE nor MSS mentions GPUs, which ought to be a perfect fit for the kind of number-crunching algorithms that the new supercomputer will be operating, and which feature in other Cray EX systems such as Frontier, Saudi Arabia's KAUST, and the LUMI supercomputer in Finland.

HPE reckons its supercomputing kit accelerates weather forecasting across the globe by delivering significant computational performance to model and simulate climate data. ®

 

[Mista]

Well, I don't really need a super computer to tell me that Singapore is always hot and humid, or whether it's either going to rain or not going to rain when it rains every other day...

 

[Han Patriot]

Well, I don't really need a super computer to tell me that Singapore is always hot and humid, or whether it's either going to rain or not going to rain when it rains every other day...

I was about to say the same thing.

 

[Song Hong]

It is a scam and monies transfer from SG taxpayer to western MNC. Worse it is going to hiring foreigners in all these engineering jobs.

Why would we want to model weather around Singapore and environ? It is more or less stable.

 

[epebble]

Well, I don't really need a super computer to tell me that Singapore is always hot and humid, or whether it's either going to rain or not going to rain when it rains every other day...

There have been about 20 near-Equatorial tropical cyclones in the past 60 years. With the climate change, there may be more in the future and more importantly, the intensity may be more, lot more. The recent Pakistan floods were unprecedented. Singapore is not stupid to get the capability to forecast future weather hazards. As a coastal city state, the state has an obligation to prevent foreseeable tragedies.

List of near-Equatorial tropical cyclones - Wikipedia

en.wikipedia.org

 

[Hamartia Antidote]

It is a scam and monies transfer from SG taxpayer to western MNC. Worse it is going to hiring foreigners in all these engineering jobs.

Why would we want to model weather around Singapore and environ? It is more or less stable.

They don't need to hire any engineers. I've worked at multiple companies with supercomputers.

However in this day and age it is surprising for companies (however...maybe government's are different) to invest in hardware when they can just offload all the hardware investment risk to some company like Amazon which has access to insane amounts of computing power in perfect climate controlled buildings and is constantly upgrading their offerings.

Amazon is going to be conjuring some Id monster someday with all their computing power.

 

[Song Hong]

They don't need to hire any engineers. I've worked at multiple companies with supercomputers.

However in this day and age it is surprising for companies (however...maybe government's are different) to invest in hardware when they can just offload all the hardware investment risk to some company like Amazon which has access to insane amounts of computing power in perfect climate controlled buildings and is constantly upgrading their offerings.

You must have never work with cloud infrastructure team. You still need a cloud infrastructure planner and implementation team to work with on prem devices. There is numerous pain points -- especially in domain such as hybrid identity authentication.

And cloud compute is at east 6x more expensive than on prem if purchase from big cloud vendors.

 

[khansaheeb]

It is a scam and monies transfer from SG taxpayer to western MNC. Worse it is going to hiring foreigners in all these engineering jobs.

Why would we want to model weather around Singapore and environ? It is more or less stable.

It has other applications too, notably weather mapping for use in hostile regions.

 

[Hamartia Antidote]

You must have never work with cloud infrastructure team. You still need a cloud infrastructure planner and implementation team to work with on prem devices. There is numerous pain points -- especially in domain such as hybrid identity authentication.

And cloud compute is at east 6x more expensive than on prem if purchase from big cloud vendors.

I'm not a cloud computer engineer.

But all of our critical systems are on Amazon. We didn't need to hire any of their engineers.

Pretty much after 9/11 when Cantor Fitzgerald got wiped out (and their financial computer systems destroyed) other financial companies thought real hard about how vulnerable their computer centers were to some kind of catastrophe. First we simply moved our machines to various remote locations but the cost of infrastructure duplication was immense...especially when it came to the supercomputers. (another issue was obviously constant continuous hacking prevention..which was a liability seen by our customers...even employees with datacenter access was a liability).

Amazon has datacenters in multiple locations and has failover to multiple locations. We have retired our hardware. Let them handle all the problems/headaches. Any questions our customers have about security get shuffled off to them..the answer being that they can certainly handle it better than we can.

You are correct that it is not cheap. Our autoscaling shuts many instances down on the weekends because we don't feel like handing Amazon free money.

Only a few of us have the power to shut certain systems down (and nobody i think has rights enough to shut all critical systems down).

Plus no angry employee can take a wrench and start smashing things and in the process taking out everything.

 

[Song Hong]

Because you guys outsource all the daily maintainence to 3rd party,

I'm not a cloud computer engineer.

But all of our critical systems are on Amazon. We didn't need to hire any of their engineers.

Pretty much after 9/11 when Cantor Fitzgerald got wiped out (and their financial computer systems destroyed) other financial companies thought real hard about how vulnerable their computer centers were to some kind of catastrophe. First we simply moved our machines to various remote locations but the cost of infrastructure duplication was immense...especially when it came to the supercomputers. (another issue was obviously constant continuous hacking prevention..which was a liability seen by our customers...even employees with datacenter access was a liability).

Amazon has datacenters in multiple locations and has failover to multiple locations. We have retired our hardware. Let them handle all the problems/headaches. Any questions our customers have about security get shuffled off to them..the answer being that they can certainly handle it better than we can.

You are correct that it is not cheap. Our autoscaling shuts many instances down on the weekends because we don't feel like handing Amazon free money.

 

[Hamartia Antidote]

Because you guys outsource all the daily maintainence to 3rd party,

Such as?

If a node crashes it gets automatically pulled and a new one started. I can see it all on the AWS CloudWatch Alarms page. Everything is basically on autopilot. We could all get neutron bombed tomorrow and that thing will still be going like Wall-E.

 

[Song Hong]

Such as?

If a node crashes it gets automatically pulled and a new one started. I can see it all on the AWS CloudWatch Alarms page. Everything is basically on autopilot. We could all get neutron bombed tomorrow and that thing will still be going like Wall-E.

Who set up new hybrid identity when a new workers come onboard -- and many others.

You cannot let users have this access privilege.

If all users have total access rights, you are sitting on top of dynamite.

 

[Hamartia Antidote]

Who set up new hybrid identity when a new workers come onboard -- and many others.

You cannot let users have this access privilege.

If all users have total access rights, you are sitting on top of dynamite.

99.9% of our workers have no access rights to the AWS location of our critical systems so I'm not sure of what the issue is. The apps they all run can certainly talk to the AWS servers but that's about the only thing they can do. It's like me looking at this PDF page..I can't shutdown the PDF servers. I have no access.

Even those of us who have the ability to access it aren't given free access 24/7. You actually have to ask for elevated access permission and give a timeframe. Some of us simply are setup to get permission by default..but we still have to ask (that way the upper management could theoretically shut us all out in an emergency by revoking the auto granting access). That's how new hires who need access rights would be handled.

If you are talking about setting up their virtual desktops then that is actually something completely independent of the critical apps.

 

[Song Hong]

99.9% of our workers have no access rights to the AWS location of our critical systems so I'm not sure of what the issue is. The apps they all run can certainly talk to the AWS servers but that's about the only thing they can do. It's like me looking at this PDF page..I can't shutdown the PDF servers. I have no access.

If you are talking about setting up their virtual desktops then that is actually something completely independent of the critical apps.

I was working as a cloud engineer planning for thousands of users. My user do not have my access levels. I onboard and remove them. I plan their domains -- something like (doman-R&D, domain-dev-opps, domain-finance). I set their access rights. I report on the compute cost.

 

[epebble]

They don't need to hire any engineers. I've worked at multiple companies with supercomputers.

However in this day and age it is surprising for companies (however...maybe government's are different) to invest in hardware when they can just offload all the hardware investment risk to some company like Amazon which has access to insane amounts of computing power in perfect climate controlled buildings and is constantly upgrading their offerings.

Amazon is going to be conjuring some Id monster someday with all their computing power.

AWS is good if you want to open a website for commerce. It is not good if you want to run highly vectorized code. A modern weather forecasting system likely uses Computational Fluid Dynamics models to predict weather patterns. This involves solving large number of equations. Typically, they will run the same algorithm over a grid thousands of data points. Ideally, best done with GPU like architecture with 128, 256 or even 1024-way parallelism. The code is typically in Vector Fortran.