So in the last post we examined whether the story of Jesus and the adulteress (or PA for pericope adulterae, because we really can’t help ourselves) had significantly more words that appear nowhere else in John, relative to its length, than passages of similar length in John 2:1-12; 5:1-11; and 6:1-15. It turns out it didn’t. But I also ended that post with a question: how sensitive is this metric? Is it actually capable of differentiating between Johannine and non-Johannine passages? In other words, does it do what we want it to do?
If I were doing this more formally, I would establish methodological principles of comparison, and then apply them to find passages outside of John to test. But we can do this roughly here with a few passages from the Synoptics, and one text more or less contemporary to John and regarding Pharisees, but non-biblical: Josephus’s Jewish War.
This passage from Mark shows Jesus meeting a challenge from the scribes, as he does in the PA. It contains 169 words, 14 of which do not appear in John.
Here is another controversy story about an issue that otherwise does not come up in John: divorce. It also has the advantage of involving the Pharisees testing Jesus, as they do in the PA. It contains 213 words, 18 of which do not appear in John.
Here Jesus comes to the defense of another accused woman against a Pharisee in a dispute over her sexual sins. It contains 273 words, 26 of which do not appear in John.
In looking up the vocabulary in the PA, I noticed that some of its unique words also appear in the longer ending to Mark. This passage was probably also a later addition to a completed gospel, but relying on earlier traditions. It contains 169 words, 25 of which do not appear in John.
Josephus, Jewish War 2.3
Here is a passage from outside the New Testament, but also addressing a controversy involving the Pharisees. In this case, it regards the ongoing sacrifices in the temple during the Jewish revolt against Rome. It contains 169 words, 52 of which do not appear in John.
I ran χ2 analyses on each of these passages against the wedding in Cana (John 2:1-12), the healing of the sick man (5:1-11), the multiplication of loaves (6:1-15), and the PA, and got the following p-values:
|Mark 3:20-30||Matt 19:1-12||Luke 7:36-50||Mark 16:9-20||Jos., JW 2.3|
|John 2:1-12||83.3%||77.5%||55.8%||2.8%||< 0.001%|
|John 5:1-11||36.9%||32.0%||19.9%||0.7%||< 0.001%|
|John 6:1-15||35.8%||30.0%||16.6%||0.3%||< 0.001%|
|John 7:53-8:11||73.8%||76.9%||97.6%||9.9%||< 0.001%|
Bolded p-values are significant (α = 5%).
On the one hand, the metric was unable to differentiate between Johannine material and passages in Mark, Matthew, and Luke. No significant p-values appear. There is no doubt these passages are not Johannine, yet looking at otherwise unique words cannot detect it. It is worth noting that the PA finds its closest comparison with Luke, since that is the other gospel in which it appears, but for all we know this is a fluke.
On the other hand, the longer ending of Mark was found consistently distinct from undisputed Johannine passages, but not from the PA. Furthermore, the metric found highly significant differences with Josephus across the board. So the metric is not completely useless, but it is not sensitive enough to separate Synoptic material from Johannine. That is, it is incapable of doing the job it’s been put to: one scholar thinks the passage sounds Johannine enough, and another thinks it sounds too Synoptic. The metric of unique words is apparently incapable of telling the difference, so a more sensitive one is needed. This is why conjectures like the one that played out in Heil’s and Wallace’s articles — that the number of unique words in such a short passage is at least partially indicative of narrative fit — need to be tested against controls.
That might not be possible with such a short passage, but in the next post we can look at some more nuanced measures that might help.