A discussion on current psionic research done in the former Soviet Union…
Date: November 18, 2006
<Jael> Hopefully all of you have read the Overview of the Russian research studies.
<[enne]> I -think- I understood most of it.
<[enne]> I have it open in the other window.
<Jael> Understanding everything about the research isn’t the important part. Being able to discuss it and learn from their methods is what this discussion will cover.
<drazak> i’m gonna take a look at it later
<drazak> i’ll log for you guys, if ya want
<Jael> Bester is here, so no worries.
<[enne]> Jael: I’m not enough here and sane to discuss anything, so I’ll probably be pretty quiet.
<Jael> that’s fine
<Jael> The authors seemed pretty careful as they presented the information, commenting about insufficient data as they addressed each study.
<Jael> Did any of you have any observations or comments about the paper?
<Kitsune> it seemed biased towards the existence of PK.
<Kitsune> I don’t think you should have a bias before the experiment.
<RainTurtle> *nods* There is some appearance of trying to back up a hypothesis, rather than test it, in the descriptions…sort of “bootstrap levitation”
<RainTurtle> (with apologies to Joel Rosenburg, for the bootstrap comment)
<Roy> it’s also dated material, a overview like this can’t be used to as evidence for or against psi phenomenon
<Roy> the experiments that they review should be replicated and refined
<Jael> Every researcher has a bias, but the experiment should counter that bias in the way it is conducted.
<RainTurtle> There isn’t enough detail given to replicate the experiments in most cases, in this paper. It’s very “high level”, as the management consultants put it
<Roy> that’s a big problem
<Jael> *nods* The material is a bit dated. However, knowing the experiments that have been tried before and the pitfalls of it are useful for future experimentation.
<Aphanas> *nods* As a meta-survey of recent “Soviet Union” research it is useful to see what to investigate further. It’s interesting to see what they are working on, but they are a bit too sparse on details of the experimental design to really have a good sense of the methodological soundness of the studies.
<Jael> The authors seemed a tad annoyed at how little information was available to them, especially in some of the studies.
<RainTurtle> It is, however, a nice article to start with, for discussion, because it doesn’t require the sort of in-depth knowledge of Physics, that most published works on this topic need
<Jael> *nods* Additionally, a serious knowledge of stats is not required.
<Jael> Let’s look at the studies in the order they are presented. First is the Distance Influence on the microcalommeter.
<Jael> Did any of you have anything specific you noticed about it or wanted to mention about it?
<Kitsune> I wish they’d actually included info on how the operators were attempting to do their thing.
<Aphanas> *nods* It is somewhat similar to the effect that Ingo Swann demonstrated for SRI initially on a magnometer. Less statistical fuzziness in an experiment where very little should affect the stability of the measuring system.
* [enne] is now known as Enne
<Jael> *nods* The system seemed pretty sensitive.
<Jael> The hit rate (6/8 for first distance, 8/13 for the second, and local was 12/18), was pretty signicant compared to the control trials (1/9)
<Aphanas> There is a similar idea explored on synthesis gas systems in Chinese literature on chi effects, in which the effect produced was spontaneous catalyzation of the system. The thing I like about experimental designs like that is that one can run a stable benchmark for days with no variance, so anomolies really do stand out – in terms of statistical significance.
<Jael> That is rather important for showing psionic effects. Without that baseline, it’s hard to show results to skeptics.
<Jael> Or to know you’re not believing something just because you want to.
<RainTurtle> Such things therefore warrant further investigation by various official (if not overly open) organizations, because large effects are not always required to produce practical results, in terms of secure communications, and less benign activities…
<Aphanas> Here’s a copy of the Chinese study of the effect of “external qi” on sythesis gas systems: http://users.eastlink.ca/~cabbages/psi/syngas.pdf
<Jael> One of the complaints I had on this study was that they didn’t discuss the level of training for the operators.
<Aphanas> It’s an interesting read… even if you skip all the stats and just read their conclusions.
<Jael> nice. Thanks
<Kitsune> jael: or what they actually did.
<Jael> Any other comments before moving on to the Electric Noise Generators?
<Aphanas> None here.
<RainTurtle> *shakes head*
<Jael> Moving on then. They had three studies here, but each of them had some problems from my read. The first had some pretty bad isolation issues.
<Jael> The first one also was missing that nice baseline that we talked about.
<Kitsune> that invalidates the whole thing.
<Aphanas> Meh… I wasn’t overly impressed by these. There are too many things that _can_ generate ordered noise in electronic systems for me to be entirely trusting of “psi” as the mechanism without a lot of proof.
<RainTurtle> Interesting idea, for future investigation, though
<Aphanas> For instance… everyone’s hard drives, processors, and cell phones are currently generating “ordered noise” that can be picked up by cheap speakers if one or both devices are not shielded well.
<Jael> The information provided for them was minimal, and therefore hard to use as an example for or against.
* Paramemetic has joined #PSC_Annex
<Kitsune> that is true. My laptop kills speakers.
<RainTurtle> hi, P
<Aphanas> Hey Paramemetic.
<Jael> Hello, Paramemetic
<Jael> Any other comments on that study before we move on to the Cellular Culture section?
<Jael> There were eight studies here, the first on chemiluminescence.
<Jael> Any comments or observations on this study?
<Kitsune> I had no idea what this one was about.
<Kitsune> seriously. Not a clue.
* eversosleepy has joined #PSC_Annex
<Aphanas> Hi, ESS
<Jael> hello, ess
<eversosleepy> howdy. What’s going on here?
<RainTurtle> (In case anyone is wondering, “Chemiluminescence” is “luminenscence (as bioluminescence) due to chemical reaction, usually at low temperatures” – Merriam-Webster’s Medical Desk Dictionary)
<RainTurtle> Hi, Everso
* Starwind has joined #PSC_Annex
* Starwind has left #PSC_Annex
<RainTurtle> Hi, STarwind
<Jael> *nods* This one was a bit odd to me too. I was impressed by the 40% difference in the amount the cell was iluminating, but that’s about all I noticed.
<Aphanas> I don’t know enough about the causes of chemiluminescence to speak intelligently about that study… no real comments on that one.
<Jael> This study would have been a tad more helpful if more info had been given.
* Aphanas does note that he didn’t see anything that gave statistical significance to the results… just the results in percentage change (which could be highly insignificant).
* Starwind has joined #PSC_Annex
<Jael> *nods* True
<Jael> Any other comments before moving on to the second Bio-PK study? The next one is affecting electrical properties of cucumber plants.
<Jael> What did folks notice about these? They had two instances they talked about in the paper.
<Kitsune> it would be nice if they described the environment changes in more detail.
<RainTurtle> At least they had a nominal “control” this time…
<Aphanas> *nods* I agree… also whether they were measuring controls in the same grounded chamber while they were running a trial.
<Kitsune> This one would be really cool if they actually included information like that.
<Jael> That would make a bit of difference.
* Vaccime has joined #PSC_Annex
<Jael> I was also wondering how long the effect would last. With the study attempting both stimulation and inhibition, that seemed a bit sketchy to me.
<Vaccime> (Hey. Keith, mind catching me up in PM or something?)
<Paramemetic> I can copy what was copied to me, I came in late too.
* Aphanas is suspicious of environmental changes that could influence plant electrophysiology… for instance: they don’t note anything about the humidity levels in the grounded chamber and whether they were controlled.
<Vaccime> That works.
<Jael> This study seemed to lack many controls that are required for quality peer-reviewed work.
<RainTurtle> *smiles slightly* And what does this tell you?
<Aphanas> They may have them in the studies… but this meta-survey doesn’t indicate it.
<Paramemetic> Either because it’s assumed, or because the meta-study has agenda bias.
<RainTurtle> If the controls actually were not there – as opposed to simply not being properly reported – it would imply that it is in the researchers’ best interest, at the time of the study, to come up with positive results
<Jael> Rain: That we don’t want to use their research methods
<Paramemetic> The cynic in me indicates the latter.
<RainTurtle> Quite so
<Paramemetic> Does anyone know if these studies were ever replicated? I don’t think the peice in question mentions, but I could’ve missed it.
<RainTurtle> It was notorious in Soviet research….if the Party does not like psionics in that particular year, suddenly none of the experiments work……
<RainTurtle> Perhaps not so much has changed, since the fall of the Iron Curtain?
<Aphanas> I’ll repost the link earlier as a study in contrasts… note the careful description of the experimental designs and control testing that are given in the Chinese study: http://users.eastlink.ca/~cabbages/psi/syngas.pdf
<Jael> heh… A critical mind is required when reading any research, since we know that folks edit studies in the West as well (*cough*Randi*cough*)
<Paramemetic> Nonsense. The Union was a bastion of proper research and scientific minding. If researchers falsified their own studies for some end or another, that’s not the fault of the Party.
<Jael> Any last comments on the plant study before moving on to the Electric Fish study?
<Jael> Alrighty… comments on this study and the researchers’ methods?
<Aphanas> Aha! They were actually trying to replicate an earlier study… this is at least a good start.
<Jael> nifty. Thanks
<Kitsune> they say that the controls were adequate, but not what they actually were.
<eversosleepy> The conditions within the tanks were kept constant, I’m hoping…
<Aphanas> *nods* I think the biggest problem with this study is simply that they were studying too complex of a mechanism. Very difficult to control all of the variables, and dubious that they did.
<Jael> That was actually one of the problems. The authors didn’t think that the controls were as adequate as the researchers.
<Jael> *as the researchers thought.
* Starwind has left #PSC_Annex
<Jael> I was also a bit surprised that they were trying to adjust the fish in the direction they go naturally. It’d be like trying to make the radiometer move faster rather than stopping.
<Paramemetic> Arguably, that’s an easier influence to enact and so simpler to demonstrate. Less drastic, less definitive, but simpler and more reasonable.
<RainTurtle> Still comparable to a psiwheel with no cover…
<Jael> oops… Aphanas corrected me. They did move it in the opposite direction. My mis-reading.
<Aphanas> “In these tests, operators started
<Aphanas> working at random moments of time. The tests were conducted with the purpose of
<Aphanas> “calming down” the fish; thus, the shifts that occurred were in the direction of an increase
<Aphanas> in the pulse intervals. At the same time, as Gurtovoy and Parkhomov pointed out, it is
<Aphanas> known that the electric fish primarily responds to changes in its environment (e.g., a change
<Aphanas> of temperature, lighting, magnetic field or to sound) with a decrease in the pulse intervals.
<Aphanas> Oops… forgot to remove the line breaks.
<Paramemetic> From my reading, and perhaps it’s just because I’m used to seeing a lot more numbers that I don’t understand in the stuff I read, but there aren’t enough numbers here to make this…useful. I’m reading ahead here and finding all of these to lack adequate control and lack description of the types of statistics employed.
<RainTurtle> I find it interesting that the animal in question appears to use its electrical pulses as sensory apparatus, rather than defensive or predatory
<Paramemetic> ANOVA? NOVA? T-Test? Z-Test? What’re we doing here? Small samples, lack of controls, and generally weak statistical data. If I can understand the math involved, it’s probably not significant.
<Aphanas> It’s really only useful as a point of reference to go find and read the actual studies, in my opinion. You couldn’t draw any conclusions from the article information. Then again… that’s what metasurveys are for (though I’ve seen better coverage of the studies in a meta-survey).
<RainTurtle> (Schroeder and Ostrander, for example?)
<Jael> *nods* I would have prefered to have more information. However, we are able to discuss good research methods vs poor research methods.
<RainTurtle> (Yep…can do a nifty, well-done paper another time)
<Jael> Any additional thoughts on the Electric Fish before moving on to the Eating Behaviour?
<Jael> Alrighty… anything new to add to the discussion with this study?
<eversosleepy> Do we know anything about the physical state of the mice before the irradiating began? Like, were they similar age, size, weight, breed?
<eversosleepy> Or were they just “mice”
<Kitsune> there is the study where they feed them, before the nasty one
<RainTurtle> It says theat they looked identical to a non-biologist….so similar size and weight. All the same breed, yes
<Jael> Excellent question, ess
<Paramemetic> Identical to a nonbiologist means virtually nothing.
<Kitsune> thing is, all white mice look the same to people, even if they are slightly different in size.
<Paramemetic> I doubt I can tell the difference between two breeds of mice of similar color and shape.
<Jael> *nods* Another example of lack of controls invalidating the energy spent on the study.
<Paramemetic> What energy spent?
<Jael> rather, invalidating the study and therefore wasting the energy of the researchers.
<RainTurtle> hmmm My bad…such details are overlooked by non-biologist types…
<RainTurtle> (Kit, in this case, is a biologist type)
<Paramemetic> It seems to me more like a science fair project than a university level research project.
<Kitsune> i would like to know more about how they calculated the weight differences. did they add up the gain of each mouse and justput that down as the groupresult/ That is kinda flawed.
<Kitsune> I know rodents.
<Aphanas> I’m more interested in this comment, personally:
<Kitsune> shame my keyboard sucks so hard.
<Aphanas> “The current study that Speransky shared with us during our trip was conducted in collaboration with Leonid M. Porvin, a specialist in electronic engineering, who, according to Speransky, developed a “technology” for achieving altered states of consciousness conducive to “distant influence.””
<Paramemetic> Cheers, it’s Russia’s Chuck.
<Aphanas> Sounds like they used a radionics-type device in the study… absolutely no information on it though.
<Jael> Any other comments before moving on to the Radiation Study?
<Paramemetic> No, but my comments on the radiation study are the same as all the others: There is not enough information here to draw a conclusion, though the data presented appears significant.
<Jael> They actually give more information in this one than the others, but it is still too little.
<Paramemetic> Comparing to a sample in which all of your mice die isn’t very significant. They should’ve used a drug with a well established LD50 to determine the deviance from an established norm.
<Jael> That sounds like it would have kept the mice alive (or at least more of them).
<Paramemetic> It’s always better to use an established control rather than a self-created one based on “well all the mice died.” Especially given an arbitrary cut off like “19 days.”
<Kitsune> this experiment is vile, FYI.
<Aphanas> Or at least establish that under normal circumstances that level of radiation for mics is 100% fatal in x number of days.
<Aphanas> *mice, rather
<Paramemetic> I’m not concerned with keeping the mice alive (sorry Kit). I’m just concerned with using a more workable control.
<RainTurtle> Similar studies had been done with wounded animals, in the past. Unless the actual experiment was looking for data specific to the effects of ionizing radiation, the experiment was a needless variant of an oft-repeated earlier series
<Paramemetic> Also…groups of 10? Please. It requires 25 to establish a normal distribution. We’re talking mice – there’s no shortage, we can whack them 25 at a time.
<Jael> That’s not necessary, Paramemetic.
<Paramemetic> Hmm. Sorry. It’s a valid point, though. 10 is hardly enough for a proper T-test, let alone a significant Z-test.
<Aphanas> Could also just as easily use an organism such as a biological culture and radiation. That one is a bit easier to establish sampling on, as you can use sections of the same culture for your controls and experimental groups.
<eversosleepy> The study on Ethnic Variations in the Chronic Pain Experience by Bates and Edwards stated they couldn’t rely on their results because one of the test groups only contained 29 subjects. I would have liked to see more in this experiment, as well.
<Paramemetic> Less readily evident results, though.
* Enne is now known as En[afk]
<Aphanas> Not if the results are 100% in the control groups, and less in the experimentals. Significance would be determined by percentage of survival. Just as evident.
<Paramemetic> It’s harder to control for the life of a biological culture. It’s quite simple to see when a group of mice has died. It’s more difficult to determine the life of the biological samples. As for the results being 100% in control groups, that’s what they’ve done /here./ It doesn’t seem workable.
<Paramemetic> For all we know, they stopped feeding the control group after they zapped them because they knew they were in a group that was going to die.
<Kitsune> mice die at the drop of a hat. They are not reliable.
<Vaccime> Thus why you need a larger sample size.
<Vaccime> To account for the psuedoentropy of the unreliability of the mice.
<Paramemetic> Thank you Vaccime.
<Aphanas> *shakes head* No… it’s not harder to control for a culture. That’s why they are used in basic biological testing so often. The only reason for mice is if you are trying to demonstrate something involving mammals (especially with ones that are similar to humans in some characteristics). Bio-cultures are very easy to use (and quite well established) for radiation effects.
<Vaccime> The problem with bio-cultures is that a very small amount of contamination can decimate the results.
<Aphanas> *nods* But they are much simpler systems, and thus you can control the possible variables more easily.
<Paramemetic> Arguably the same results for mice. We could use a larger sample for that, too…
<Paramemetic> However, the trick is that we’re somewhat experimenting on the potential effects on biological macrosystems.
* MobC|Work is now known as MagicianOfBlackChaos
<Paramemetic> Killing a bioculture isn’t sufficiently more complicated than altering the electrical output of a vegetable.
<Paramemetic> This research is aimed directly at healing effects, not at mere demonstration of the existence of psychokinesis – that’s a basic premise of the study.
<Jael> A larger sample size is needed, most certainly. And mice are known for dying easily (per Kit, our resident expert on the critters), so this study has issues on multiple levels.
<Vaccime> Growth rate. Growth rate is easily tracked, and well understood, if we do want to use biocultures.
<Kitsune> since that hasn’t been sufficiently proven, it is a false basis.
<Aphanas> *waggles fingers* Then you should be testing the sub-systems that you suspect would be involved. For instance… I have a study that illustrates this… one sec.
<Paramemetic> It was (unfortunately, for Singer-ites) necessary to have a sick animal in this study.
<Paramemetic> The study was determining the ability of PK influence to affect healing in cultures that were effectively doomed.
<Kitsune> <Kitsune> since that hasn’t been sufficiently proven, it is a false basis.
<Paramemetic> It doesn’t need to be sufficiently proven to run a study on it. We do research all the time based on assumed premises.
<Paramemetic> Gravity, for instance.
<Paramemetic> Or…I’m sure Vaccime could provide a better example. Ah, all psychological research.
* Jael pulls the convo back to the study at hand.
<Vaccime> Go ahead, Jael.
<Paramemetic> We assume the existence of pathologies when we are doing research on them. We assume that MDD exists, when researching effects in depressed patients. We needn’t prove it every time we conduct a study.
* En[afk] is now known as Enne
<Jael> It is established that this study was flawed on the type of critter as well as the study size. Are there other concerns with this study?
<Jael> Concerns or ‘hey they did this right’?
<Vaccime> There doesn’t necessarily have to be any other concerns. In my mind, the two of those together are rather damning to the study.
<Paramemetic> I don’t think we can say that last bit about /any/ of these studies without more information on them, which isn’t forthcoming.
<Jael> Vaccime: indeed
<Aphanas> [Side comment… example study of psi/qi on liposome phase behavior in cells (which is a factor in healing): http://users.eastlink.ca/~cabbages/psi/Q1.pdf]
<RainTurtle> Presuming that the researchers did the basic “review of the literature” before starting their experiment, they would know that many similar studies had already been done – in their own country and elsewhere. I suspect that perhaps the objective was not quite as stated, in that one.
<Jael> On to the next study… Interference of Math Processing. Were there good things about this study? The authors pointed out some of the flaws in their survey.
<Kitsune> they failed to define “sensitive”
<Jael> *nods* They commented about two of the people and mentioned their percentage, but like has already been pointed out, percentages can be misleading.
<RainTurtle> From where I stand, they did not demonstrate any real effect at all, in this one,…at least as it is described in the article
<Aphanas> And there is a real problem with the lack of proper double-blind controls mentioned… did the people administering the tests to subjects know which control groups they belonged to? If so, that could drastically skew the results due to unintentional communication (body language, etc).
<Jael> The second to last study, the EEG study, had a few more trials… any comments on this study?
<Paramemetic> It did not appear significant to me, but I’ve never seen EEG results in that format.
<Paramemetic> If the subjects were even so much as hinted that something should be happening, we should be expecting beta wave activity…
<Jael> I was wondering about how they kept the people from changing brain states ‘just because’.
<Aphanas> It’s a much more thorough description of the experimental design, however… I’m much happier with their description here than most of the rest of the article.
<Paramemetic> Complete suppression of alpha state?
<Paramemetic> I would really, really need to know what their electrode placement scheme was.
<Aphanas> They said that beta had no significant changes… apparently they did record it.
<Paramemetic> Things like “opening your eyes” can significantly reduce the alpha wave readings.
<Paramemetic> It’s my opinion not that the researchers had little understanding, but that the authors of this study had little understanding, of how EEGs work.
<Aphanas> Hmm… If we presume consistency between subjects in electrode placement, it’s at least plausible that the results are significant.
<Paramemetic> I’m unsure…with alpha waves being as variable as they are…
<Paramemetic> Also, with things like brainwave, we have to factor alertness and arousal factors including time of day, diet, sleep prior, rest state, comfort levels, stimuli in the room…even so much as a bright painting on a wall…
<Aphanas> Yup. Would be nice if they discussed control of input during the study.
<Jael> Those are the type of things I was wondering about. I’m not horribly familiar with EEGs, but the brainwaves seem like changable things.
<Aphanas> Edwin May is a PhD in Physics by background. It’s possible that he’s not as familiar with EEG readings as a standard parapsychologist.
<Jael> Anything else on this study before moving to the last one on Reaction Time?
<Aphanas> (from a psychology/neuroscience background.
<Aphanas> *shakes head*
<Jael> This last one on Reaction Time is a preliminary study. Any comments on this?
<Paramemetic> Yes. “What?”
<Kitsune> I had no idea what was going on in this.
<Paramemetic> To break from technical terminology, my understanding is that they were basically nerve-jamming the mark to affect reaction speed?
<Paramemetic> The idea being to jam the ability to press the button quickly, or summat?
<Jael> It appears that they are wanting to increase the time it takes for someone to click a button after seeing a light. They are using the right hand (issue… what if the person is left handed or ambidexterous?).
<Vaccime> (THen the reaction time would be slow to begin with, or slower if affected.)
<Vaccime> If they’re measuring the delta, of course.
<Jael> indeed, the reaction would change. But their baselines would then be off.
* eversosleepy has quit IRC
<Paramemetic> They also pull out these random percentages that seem to be related to…well, nothing, and then they use arbitrary terms of “significance” that are effectively useless…if I were a science teacher and this were given me as an abstract, I’d reject it on principle.
<RainTurtle> “In discussions with Dr. Lebedeva in our laboratory, we identified some methodological improvements for a formal replication” ‘nff said
<Vaccime> Have them do ten runs unaffected, and 10 affected. If you’re graphing according to average change between runs, then it is able to be standardized.
<Paramemetic> Does the subject know they are in an experiment about reaction time? If so, hello performance anxiety.
<Jael> I get that just at the eye doctor’s
<Vaccime> Keith: I imagine, given the prompt “push this button when the light comes on”, they could be able to infer that it is reaction time.
<Paramemetic> Phrasing is everything.
<Roy> ideally they’d run many trials
<Roy> any sort of effect caused by performance anxiety should even out
<Paramemetic> Seriously. There is difference between populations who are told “your results are being recorded” and who aren’t told that, even though it’s implied that results are recorded.
<Jael> not surprising, that.
<Paramemetic> It would be the same here.
<Roy> something akin to the Hawthorne effect
<Aphanas> *nods* If done consistently… it shouldn’t be an issue, however.
<Aphanas> Presuming you have enough subjects and enough trial runs (properly randomized, of course).
<Aphanas> The measured effect of the influence might be closer to control trials under pressure… but you should still be able to demonstrate a change. Many psych studies have to work around anxiety-inducing scenarios.
<Aphanas> *to control runs…
<Jael> Thus concludes this discussion. This has been an example of some poorly reported research (being generous) and some poorly done research.
<Jael> When I first found the article, I had hoped it would be a peek into what was going on in Russia. Instead, I saw this as an opportunity to discuss research methods. That made it a good first discussion because of the way non-scientists tend to look at research.
<Jael> Actually, does anyone have any further comments about the article?
<Vaccime> Not I.
<Jael> (sorry… got ahead of myself.)
<RainTurtle> In spite of the ….flaws….in the report, it at least demonstrates that there is research being done.
<Jael> very true
<Paramemetic> Or rather, there was.
<Vaccime> That begs the question though, is bad research better than no research?
<Paramemetic> It doesn’t beg the question.
<Paramemetic> But it does lead to the question being asked.
<Vaccime> If anything, it may even be counterproductive to have horribly stated experiments being done, as it detracts from parapsych as science.
<Aphanas> *nods* The authors were rather critical of the Russian methodologies at several points… makes me wonder if the problem could have been that they could only report with the information they were given, but were already obliged to publish the results of their trip. Seen that sort of thing happen in academia before…
<RainTurtle> Granted, Muffin….however, it demonstrates to the kids that they are not the first people ever to think of research in this field
<Vaccime> That is true.
<Jael> Thanks for attending. I hadn’t realised how long the discussion would go. I appreciate your patience and participation.
<Jael> We’ll be discussing “The Possible Role of Intention, Attention and Expectation in Remote Viewing” by McMoneagle and May on December 9, same time, same place.