Discussion Closed This discussion was created more than 6 months ago and has been closed. To start a new discussion with a link back to this one, click here.

Linux CFD is 10X faster than Windows-10 CFD

Please login with a confirmed email address before reporting spam

Ok, I can’t really claim COMSOL-CFD is always 10X faster under Linux than under Windows 10, but I can say large k-w models of compressible non-isothermal high-velocity flows in rotating machinery run more than 10 times faster under Linux than they do under Windows 10 on a computer based on two E5-2697v4’s (Q1’16, 512 GB DDR4/2400, 16 cores in each processor, 40MB cache) on a motherboard using the C612 chip set and with a SSD.

We have been screaming to COMSOL support for more than 2 years that we need someone at COMSOL to take seriously the problem that COMSOL-CFD-Mixer can’t use more than a very small fraction of the available CPU resources in modern high-end multi-core machines under Windows 10, at least for large k-w rotating machinery models (and probably for other types of CFD models too). We have demonstrated this serious deficiency on at least 7 machines and hundreds of widely differing cases, with every imaginable change in solver parameters.

No one really took our problem seriously, though we know many COMSOL users have had the same problem (some have asked us for guidance !).

We noted that several months ago, after a Windows 10 update, several machines which were already terribly slow, started running even slower, by more than another factor or 3 !

Finally, a month ago, Bjorn Sjodin, suggested in a phone call that we try Linux. He was aware of some serious issues using modern multi-core processors under Windows 10. He said some code modules were generated using different compilers for running under Linux than under Windows, and they might be better able to utilize available resources. So, in desperation, we got Linux going on one of our machines (as described above). There was a learning curve. We still have a lot to learn. But we got COMSOL to issue the needed license, and yesterday we got our first results.

10 TIMES FASTER !!

We ran a large model (more than 1M meshcells) under 5.2a because that is what we’ve been using for these large cases – because 5.2a handled our cases involving very high shear stresses in transonic flows better than 5.3a did.

We did not rebuild the model or the mesh, or change any initial conditions or model parameters. We didn’t change any solver parameters. We ran a single instance with everything exactly as what was previously working best under Windows 10 – direct solver, PARDISO, 6 iterations from the same initial conditions, no changes in damping or initial CFL number, or anything else.

A couple months earlier on this same computer under Windows 10, this model took 5.1 hours to run. Now, under Linux, this model runs in 30.1 minutes.

THANK YOU BJORN SJODIN !!

We still have a lot to learn about Linux. We have to get Linux versions of all the other software we need to have at our finger tips on this machine. We will soon install COMSOL 5.3a and get this model going under it (may take a while, as we’ll have to change some meshing and solver parameters…). We expect we’ll again be ecstatic. We’ve previously seen little difference in solver speed between 5.2a and 5.3a on any machine under Windows 10 for the same number of iterations. We expect we’ll see the same under Linux. (We have seen it usually takes more iterations under 5.3a to get to the same convergence, but that’s another, and comparatively minor, issue.)

A few months ago we bought the fastest computer we could justify: Two Xeon gold 6138’s, Q3’17; 20 cores, 2GHz, 3.7GHz turbo, 6 memory channels; 22MB cache; 512GB DDR4/2666. Supermicro X11DAi-N Dual Intel Skylake-SP Xeon WS board. We ran the same CFD test case on this computer, under Windows 10, and it ran almost exactly as slow as on the earlier computer (the one with dual 2697’s) even though this computer runs most standard benchmarks 20-30% faster.

We’re now optimistic that we can switch from Windows 10 to Linux on this Xeon Gold machine and see another 20-30% increase in the speed of COMSOL-CFD. We’ll probably try that at some point, in a month or so, after we’re fully comfortable with Linux. But for now, we’re very happy with a factor of 10 increase in solver speed, and optimistic that we’ll be able to move to 5.3a under Linux and take advantage of its improvements in geometry handling, selections, and meshing, and eventually figure out some remaining solver issues.

I suppose some moderator will soon take this post down because it says COMSOL-CFD-Mixer is not compatible with Windows 10 on modern high-end machines. But I hope not. I hope COMSOL highlights it. There are probably hundreds of users out there (maybe thousands) hoping someone will tell them how to get COMSOL-CFD to run large problems effectively on modern high-end machines.

The answer is (relatively) simple. Switch from Windows 10 to Linux !! Then COMSOL-CFD works !!

David Doty, PhD


4 Replies Last Post 19.07.2018, 17:05 GMT-4
Bjorn Sjodin COMSOL Employee

Please login with a confirmed email address before reporting spam

Posted: 6 years ago 28.06.2018, 11:34 GMT-4
Updated: 6 years ago 28.06.2018, 11:35 GMT-4

Hi David,

I'm happy to hear that you got this level of speed improvement. Good news is that this particular performance problem in Windows will go away in the next version that we plan to release this fall.

Best regards, Bjorn

Hi David, I'm happy to hear that you got this level of speed improvement. Good news is that this particular performance problem in Windows will go away in the next version that we plan to release this fall. Best regards, Bjorn

Please login with a confirmed email address before reporting spam

Posted: 6 years ago 28.06.2018, 15:30 GMT-4

Great ! We're looking forward to it ! I guess we'll wait how that works on our new Xeon Gold machine before trying to swith it to Linux. Thanks, David

Great ! We're looking forward to it ! I guess we'll wait how that works on our new Xeon Gold machine before trying to swith it to Linux. Thanks, David

Jim Freels mechanical side of nuclear engineering, multiphysics analysis, COMSOL specialist

Please login with a confirmed email address before reporting spam

Posted: 6 years ago 28.06.2018, 21:17 GMT-4

Bjorn and others at COMSOL know and understand that I am a Linux-only user for many reasons starting about 1994 when Windows-3.1 provided me with the blue screen of death. Folks I can also tell you that the Linux desktop has much more to chose from. For those occasions where I absolutely have to have Windows, I run it inside my Linux box on a vmware virtual machine. It essentially becomes another Linux application. Examples of Windows-only applications are Quicken, Microsoft Office (Word, Powerpoint, Excel), and COMSOL application builder. There are several equivalents that run under Linux natively, but not as good for Quicken and Office. There is no substitute for COMSOL application builder. One of the primary reasons (high on the filter list for screening out other software) initially for me in choosing COMSOL (FEMLAB 3.0 at the time) was that it ran on Linux. If it had not done that, I probably would not have started using COMSOL in the first place. I am very thankful that was not the case ! A factor of 10 is likely due to a programming bug of some kind either in COMSOL or support libraries. However, there is indeed a reason why Linux is the only operating system that runs all 500 of the top 500 fastest computers in the world: https://www.zdnet.com/article/linux-totally-dominates-supercomputers/ One should not be afraid to try Linux as you have found. It is very much improved and much easier to install and configure than earlier versions.

-------------------
James D. Freels, Ph.D., P.E.
Bjorn and others at COMSOL know and understand that I am a Linux-only user for many reasons starting about 1994 when Windows-3.1 provided me with the blue screen of death. Folks I can also tell you that the Linux desktop has much more to chose from. For those occasions where I absolutely have to have Windows, I run it inside my Linux box on a vmware virtual machine. It essentially becomes another Linux application. Examples of Windows-only applications are Quicken, Microsoft Office (Word, Powerpoint, Excel), and COMSOL application builder. There are several equivalents that run under Linux natively, but not as good for Quicken and Office. There is no substitute for COMSOL application builder. One of the primary reasons (high on the filter list for screening out other software) initially for me in choosing COMSOL (FEMLAB 3.0 at the time) was that it ran on Linux. If it had not done that, I probably would not have started using COMSOL in the first place. I am very thankful that was not the case ! A factor of 10 is likely due to a programming bug of some kind either in COMSOL or support libraries. However, there is indeed a reason why Linux is the only operating system that runs all 500 of the top 500 fastest computers in the world: https://www.zdnet.com/article/linux-totally-dominates-supercomputers/ One should not be afraid to try Linux as you have found. It is very much improved and much easier to install and configure than earlier versions.

Please login with a confirmed email address before reporting spam

Posted: 6 years ago 19.07.2018, 17:05 GMT-4

Appreciate your positive perspective on Linux, James. I mentioned we'd been staying with 5.2a because we'd never gotten 5.3a to handle our transonic high-shear-stress flows as well as 5.2a did, but we were going to try 5.3a again, on our new Linux Xeon Gold machine, because of some of its other improvements. Well, we finally got to that, and were very disappointed.

We had a large CFD-Mixer problem that we had worked on in 5.3a a fair bit 5 months ago on a Windows machine (dual 2697's) before abandoning it and going back to our earlier 5.2a model. We ran this same model on our Linux machine in 5.3a, and it was even slower than on the Windows machine in 5.3a - maybe ~ 1/20 the speed of 5.2a under Linux.

So for now, we're staying with 5.2a. Really hoping the next version of COMSOL-CFD-Mixer does address the serious performance problems we're seeing in all of our cases thus far except for 5.2a under Linux.

David

Appreciate your positive perspective on Linux, James. I mentioned we'd been staying with 5.2a because we'd never gotten 5.3a to handle our transonic high-shear-stress flows as well as 5.2a did, but we were going to try 5.3a again, on our new Linux Xeon Gold machine, because of some of its other improvements. Well, we finally got to that, and were very disappointed. We had a large CFD-Mixer problem that we had worked on in 5.3a a fair bit 5 months ago on a Windows machine (dual 2697's) before abandoning it and going back to our earlier 5.2a model. We ran this same model on our Linux machine in 5.3a, and it was even slower than on the Windows machine in 5.3a - maybe ~ 1/20 the speed of 5.2a under Linux. So for now, we're staying with 5.2a. Really hoping the next version of COMSOL-CFD-Mixer does address the serious performance problems we're seeing in all of our cases thus far except for 5.2a under Linux. David

Note that while COMSOL employees may participate in the discussion forum, COMSOL® software users who are on-subscription should submit their questions via the Support Center for a more comprehensive response from the Technical Support team.