Sorry Geoff I do not understand the point of the question.
But if you mean what happens if the listener perceives an improvement or none with both the placebo/duplicate and wotsit and no measurement differences, then its fair to say there was no improvement.
If it turns out there is an improvement with both the placebo/duplicate and wotsit we have a cognition skew result.
My point that I think your now going to argue on semantic is this; there is no guarantee a listener will respond to any of what I outlined in previous post but you must have checks in place.
Therefore it is plausible and still acceptable for a listener to not perceive an improvement with both the placebo and the real product.
For that listener adding the products or a false product has no effect on them.
The next step could be argued you need a larger listener pool to validate the initial result/s.
What you really want for the product to be deemed a success is as follows;
- Preferential improvement with only the genuine product and not also placebo/duplicate, if both it is a fail for that listener.
- Both product and placebo/duplicate not improving preference over original room but maybe measurement differences are identified and then this may come down to sensitivity of the listener for the product effect.
- At least one listener who passes the above (so if it is a fail X more listeners are required to create a small but acceptable amount of listeners for the test).
- Aside from the listeners there are the measurement/modelling that also may provide insights and may show some effects, but ideally you need both measurement and modelling to align.
This is really oversimplifying it, but it is pretty obvious how using real product/duplicate-placebo/monitor listener/measurement and modelling all work together to provide accurate positive or negative test result.
You do understand also my point where without this you can have the argument cycle where the measurements do not change and yet the listener says they have an improvement if only using the real product?
Those against the product will state measurements prove a product fail is happening and improvement is psychoacoustic/cognition related, while others will state that we do not know yet how to measure everything and the product works because the listeners perceived an improvement.
If the test is done blind without using placebo/duplicates then again those supporting the product will argue it is an unfair listener situation because the listener is put into an awkward situation (blindfolded or whatever due to no placebo) possibly comparable to the ABX that nearly everyone fails.
Bear in mind regarding the placebo/duplicate test the listener is made aware that what they see added could be either a real or fake product, this then makes the listener more comfortable on a dbt/sbt and also balance out skewing we possibly would had seen otherwise for reasons mentioned.
mOFC can also be monitored to see if adding the products even with the listener aware of placebo is boosting a product expectation that would also skew results.
Again I am oversimplifying but the cycle of argument/caveat must be removed if this test is to have any true value.
Again this comes back to then using real product/duplicate-placebo/monitor listener/measurement and modelling all working together.
What would be your proposal to overcome the argumentative scenario?
Edit:
I just noticed you mentioning other products such as cables, can we please take that to a seperate thread or PM as it has no relevance to this specific test, unless you are looking to make a case on semantics and or process.
The unfortunate side effect is to draw attention away from the specifics associated with the actual test for the bowls that we were discussing recently.
Thanks
Orb