I’ve got an ASRock Z97 Extreme6 mobo.  I put in a Samsung 970 EVO NVMe M.2 1tb card.  I cloned my SSD to it and then ran tests (it booted fine from it using Windows Bootloader in BIOS).  Bootup was unchanged in terms of speed, whether from cold, hibernate, or restart.  Tests using Samsung magician showed utterly no major differences.  Essentially the only advantage is another SSD without using an SATA bay.

PCIe 3.0 x4 will yield a top transfer speed of 32GB/s, however, there is no M.2 SSD that comes anywhere near that yet.

We have plenty of articles that speak to the difference between a hard drive and SSD. It is a no brainer that you will observe massive and obvious system improvements from ANY SSD, solely because of the significant decrease in access times. Whether you need a SATA 3 SSD or PCIe, however, is something you must determine in your own needs.

I am posting here for answers to questions I get from OTHER places on the internet. Please do not take my text as refering only to what is offered here. There is only one other place I have found that has provided accurate info that begins to touch on PCIe slot usage-speed limitations, and like here, that site’s primary purpose is not about PCIe education.

That said,
From the article you linked above: “ASRock has implemented two Ultra M.2 PCIe 3.0 X3 and each is one day capable of seeing a high of 32GB/s transfer speed”, did you mean PCIe 3.0 x4 and 3.2GB/s?

And back to the puzzle in my head:
I really like how you detail your setup to the reader.
For my budget, I can not touch the x99/i7-5000, so I am left puttering around in low-budgetland, either hoping to see a review that actually ran a similar set up as I hope to get or keeping my fingers crossed when it comes to spending more for something that might not run at its full potential since I do not understand the limits of the items I will buy.
For mobo info, it is very confusing to read: 2 PCIe x16 {well what version are either of them?)
Then wait, no one can use both of them at x16. {ahhh, the “by number” is used interchangeably to mean physical size and also speed … I think? … confusing!}
And if slot A {x16 }and slot B {x16} are used then slot C {x16 size} can only only be used at x4 {and that may or may not slow slots A or B down to half the speed they could run just before slot C was populated}.
But wait: if slot C is populated, then slots D, E and F {all x1 size} can not be used and maybe some other ports that are not even PCIe …
Gah. There must be a formula and there are obviously limitations, but they seem to be complex. And then throw in marketing which blankets the facts with half truths and any chance of decoding this stuff becomes … well here I am ranting in the very place I am beginning to glean answers.

I also just learned (correct me if wrong): due to my budget limits I probably should just forget the upper end SSD that would need the use of PCIe to run at full potential. I should stick with a meh-middling SSD that plugs into and runs at about SATA III max. This will still give me about almost 4 times faster speed than a SATA HDD. Did I get it right when I pegged HDDs (7200s) as only using about 1/4 of SATA III speed limit? If so, then I will be content at my modest speed increase over my late-2005 pc which employed PCIe and sATA when they were new.

Jim, we are learning as we go also. Take a look at our latest report on the Extreme 11 X99…just posted. I actually went out and bought a second GTX 770 to SLI both (x16/x16). On top of that, I have an Intel DC P3700 NVMe SSD running at PCIe 3.0 x4, two Samsung XP941 PCIe 2.0 x4 SSDs running in the Ultra M.2 PCIe X4 slots, 8 SAS HGST 12Gbps SSDs RAIDED along with a notebook SSD or two and all function correctly without any difficuly whatsoever.

https://www.thessdreview.com/featured/asrock-x99-extreme11motherboard-review-6-1gbs-transfer-speed-sound/

If you are set on the Z97 chipset, I can ask Sean to jump in as he is running that Z97 system right now. He may… have dual cards SLI’d and I also know he has a few other PCIe peripherals he may be able to pop in to see how things fare.

I scour the internet looking for details of this.
I am hoping I can easily get a $80-$130 Z97/H97 mobo which will run an i5 4590 and GTX 970 and a HDD now. And in about six months also accept an SSD that will not be limited to SATA.
It looks like I “do not have to worry” because the 970 can not even push the limits of PCIe 3.0 x8 so it does not matter if the SSD knocks the PCIe 3.0 x16 holding 970 down to x8 and at the same time the SSD will be able to get at least x4.

Most mobos seems to have one PCIe 3.0 x16 and then the rest is PCIe 2.0. And so really the second GPU running at x8 is PCIe 2.0 x8 (which is PCIe 3.0 x4 speed). Do the current fastest GPUs push the limit of PCIe 2.0 x8?

The z97 extreme9 manual pdf you linked above seems to indicate there are two full speed PCIe 3.0 x16 slots available, but after all the multi-GPU-hype dust settles … it is hard to determine what effect, if any, installing a m.2 device to the ultra m.2 interface. Does it knock down one of the PCIe 3.0 x16 slots? Are the PCIe 3.0 x16 slots already knocked down by having dual GPUs?

What the heck is the formula for figuring this stuff out?
The best hazy info I get seems to indicate for the average October 2014 Z97/H97 mobo is: 1 current GPU can run at x16, add a another current GPU or something else that can use PCIe and that first GPU then sees x8 (which is not a bottleneck). Have two GPUs and another PCIe user (m.2 SSD maybe??) and the speeds are x8 for the first GPU, x4 for the second GPU and x4 for the third device.
There seems to be some variations which may be decided by the manufacturer? Or is some inherent limit of the current Intel Core CPU and the Z/H97 chipset?

Please feel free to correct my ignorance and to answer any relevant questions left after corrections. It was only earlier today that I learned it does not matter how much SATA is populated … it will not affect any PCIe activities.