Tachyon /Users/simon/.local/share/uv/python/cpython-3.15.0-macos-aarch64-none/lib/python3.15/difflib.py
6205
Self Samples
13711
Cumulative
25
Lines Hit
47.11%
% of Total
1916
Max Self
5859
Max Total
Intensity:
Cold Hot
Self Time
Total Time
Show All
Hot Only
Heat
Specialization
Line
Self
Total
Code
1
"""
2
Module difflib -- helpers for computing deltas between objects.
3
4
Function get_close_matches(word, possibilities, n=3, cutoff=0.6):
5
Use SequenceMatcher to return list of the best "good enough" matches.
6
7
Function context_diff(a, b):
8
For two lists of strings, return a delta in context diff format.
9
10
Function ndiff(a, b):
11
Return a delta: the difference between `a` and `b` (lists of strings).
12
13
Function restore(delta, which):
14
Return one of the two sequences that generated an ndiff delta.
15
16
Function unified_diff(a, b):
17
For two lists of strings, return a delta in unified diff format.
18
19
Class SequenceMatcher:
20
A flexible class for comparing pairs of sequences of any type.
21
22
Class Differ:
23
For producing human-readable deltas from sequences of lines of text.
24
25
Class HtmlDiff:
26
For producing HTML side by side comparison with change highlights.
27
"""
28
29
__all__ = ['get_close_matches', 'ndiff', 'restore', 'SequenceMatcher',
30
'Differ','IS_CHARACTER_JUNK', 'IS_LINE_JUNK', 'context_diff',
31
'unified_diff', 'diff_bytes', 'HtmlDiff', 'Match']
32
33
from heapq import nlargest as _nlargest
34
from collections import namedtuple as _namedtuple
35
from types import GenericAlias
36
lazy from _colorize import can_colorize, get_theme
37
38
Match = _namedtuple('Match', 'a b size')
39
40
def _calculate_ratio(matches, length):
41
if length:
42
return 2.0 * matches / length
43
return 1.0
44
45
class SequenceMatcher:
46
47
"""
48
SequenceMatcher is a flexible class for comparing pairs of sequences of
49
any type, so long as the sequence elements are hashable. The basic
50
algorithm predates, and is a little fancier than, an algorithm
51
published in the late 1980's by Ratcliff and Obershelp under the
52
hyperbolic name "gestalt pattern matching". The basic idea is to find
53
the longest contiguous matching subsequence that contains no "junk"
54
elements (R-O doesn't address junk). The same idea is then applied
55
recursively to the pieces of the sequences to the left and to the right
56
of the matching subsequence. This does not yield minimal edit
57
sequences, but does tend to yield matches that "look right" to people.
58
59
SequenceMatcher tries to compute a "human-friendly diff" between two
60
sequences. Unlike e.g. UNIX(tm) diff, the fundamental notion is the
61
longest *contiguous* & junk-free matching subsequence. That's what
62
catches peoples' eyes. The Windows(tm) windiff has another interesting
63
notion, pairing up elements that appear uniquely in each sequence.
64
That, and the method here, appear to yield more intuitive difference
65
reports than does diff. This method appears to be the least vulnerable
66
to syncing up on blocks of "junk lines", though (like blank lines in
67
ordinary text files, or maybe "<P>" lines in HTML files). That may be
68
because this is the only method of the 3 that has a *concept* of
69
"junk" <wink>.
70
71
Example, comparing two strings, and considering blanks to be "junk":
72
73
>>> s = SequenceMatcher(lambda x: x == " ",
74
... "private Thread currentThread;",
75
... "private volatile Thread currentThread;")
76
>>>
77
78
.ratio() returns a float in [0, 1], measuring the "similarity" of the
79
sequences. As a rule of thumb, a .ratio() value over 0.6 means the
80
sequences are close matches:
81
82
>>> print(round(s.ratio(), 2))
83
0.87
84
>>>
85
86
If you're only interested in where the sequences match,
87
.get_matching_blocks() is handy:
88
89
>>> for block in s.get_matching_blocks():
90
... print("a[%d] and b[%d] match for %d elements" % block)
91
a[0] and b[0] match for 8 elements
92
a[8] and b[17] match for 21 elements
93
a[29] and b[38] match for 0 elements
94
95
Note that the last tuple returned by .get_matching_blocks() is always a
96
dummy, (len(a), len(b), 0), and this is the only case in which the last
97
tuple element (number of elements matched) is 0.
98
99
If you want to know how to change the first sequence into the second,
100
use .get_opcodes():
101
102
>>> for opcode in s.get_opcodes():
103
... print("%6s a[%d:%d] b[%d:%d]" % opcode)
104
equal a[0:8] b[0:8]
105
insert a[8:8] b[8:17]
106
equal a[8:29] b[17:38]
107
108
See the Differ class for a fancy human-friendly file differencer, which
109
uses SequenceMatcher both to compare sequences of lines, and to compare
110
sequences of characters within similar (near-matching) lines.
111
112
See also function get_close_matches() in this module, which shows how
113
simple code building on SequenceMatcher can be used to do useful work.
114
115
Timing: Basic R-O is cubic time worst case and quadratic time expected
116
case. SequenceMatcher is quadratic time for the worst case and has
117
expected-case behavior dependent in a complicated way on how many
118
elements the sequences have in common; best case time is linear.
119
"""
120
121
def __init__(self, isjunk=None, a='', b='', autojunk=True):
122
"""Construct a SequenceMatcher.
123
124
Optional arg isjunk is None (the default), or a one-argument
125
function that takes a sequence element and returns true iff the
126
element is junk. None is equivalent to passing "lambda x: 0", i.e.
127
no elements are considered to be junk. For example, pass
128
lambda x: x in " \\t"
129
if you're comparing lines as sequences of characters, and don't
130
want to synch up on blanks or hard tabs.
131
132
Optional arg a is the first of two sequences to be compared. By
133
default, an empty string. The elements of a must be hashable. See
134
also .set_seqs() and .set_seq1().
135
136
Optional arg b is the second of two sequences to be compared. By
137
default, an empty string. The elements of b must be hashable. See
138
also .set_seqs() and .set_seq2().
139
140
Optional arg autojunk should be set to False to disable the
141
"automatic junk heuristic" that treats popular elements as junk
142
(see module documentation for more information).
143
"""
144
145
# Members:
146
# a
147
# first sequence
148
# b
149
# second sequence; differences are computed as "what do
150
# we need to do to 'a' to change it into 'b'?"
151
# b2j
152
# for x in b, b2j[x] is a list of the indices (into b)
153
# at which x appears; junk and popular elements do not appear
154
# fullbcount
155
# for x in b, fullbcount[x] == the number of times x
156
# appears in b; only materialized if really needed (used
157
# only for computing quick_ratio())
158
# matching_blocks
159
# a list of (i, j, k) triples, where a[i:i+k] == b[j:j+k];
160
# ascending & non-overlapping in i and in j; terminated by
161
# a dummy (len(a), len(b), 0) sentinel
162
# opcodes
163
# a list of (tag, i1, i2, j1, j2) tuples, where tag is
164
# one of
165
# 'replace' a[i1:i2] should be replaced by b[j1:j2]
166
# 'delete' a[i1:i2] should be deleted
167
# 'insert' b[j1:j2] should be inserted
168
# 'equal' a[i1:i2] == b[j1:j2]
169
# isjunk
170
# a user-supplied function taking a sequence element and
171
# returning true iff the element is "junk" -- this has
172
# subtle but helpful effects on the algorithm, which I'll
173
# get around to writing up someday <0.9 wink>.
174
# DON'T USE! Only __chain_b uses this. Use "in self.bjunk".
175
# bjunk
176
# the items in b for which isjunk is True.
177
# bpopular
178
# nonjunk items in b treated as junk by the heuristic (if used).
179
180
self.isjunk = isjunk
181
self.a = self.b = None
182
self.autojunk = autojunk
183
499
self.set_seqs(a, b)
184
185
def set_seqs(self, a, b):
186
"""Set the two sequences to be compared.
187
188
>>> s = SequenceMatcher()
189
>>> s.set_seqs("abcd", "bcde")
190
>>> s.ratio()
191
0.75
192
"""
193
194
self.set_seq1(a)
195
499
self.set_seq2(b)
196
197
def set_seq1(self, a):
198
"""Set the first sequence to be compared.
199
200
The second sequence to be compared is not changed.
201
202
>>> s = SequenceMatcher(None, "abcd", "bcde")
203
>>> s.ratio()
204
0.75
205
>>> s.set_seq1("bcde")
206
>>> s.ratio()
207
1.0
208
>>>
209
210
SequenceMatcher computes and caches detailed information about the
211
second sequence, so if you want to compare one sequence S against
212
many sequences, use .set_seq2(S) once and call .set_seq1(x)
213
repeatedly for each of the other sequences.
214
215
See also set_seqs() and set_seq2().
216
"""
217
218
if a is self.a:
219
return
220
self.a = a
221
self.matching_blocks = self.opcodes = None
222
223
def set_seq2(self, b):
224
"""Set the second sequence to be compared.
225
226
The first sequence to be compared is not changed.
227
228
>>> s = SequenceMatcher(None, "abcd", "bcde")
229
>>> s.ratio()
230
0.75
231
>>> s.set_seq2("abcd")
232
>>> s.ratio()
233
1.0
234
>>>
235
236
SequenceMatcher computes and caches detailed information about the
237
second sequence, so if you want to compare one sequence S against
238
many sequences, use .set_seq2(S) once and call .set_seq1(x)
239
repeatedly for each of the other sequences.
240
241
See also set_seqs() and set_seq1().
242
"""
243
244
if b is self.b:
245
return
246
self.b = b
247
self.matching_blocks = self.opcodes = None
248
self.fullbcount = None
249
499
self.__chain_b()
250
251
# For each element x in b, set b2j[x] to a list of the indices in
252
# b where x appears; the indices are in increasing order; note that
253
# the number of times x appears in b is len(b2j[x]) ...
254
# when self.isjunk is defined, junk elements don't show up in this
255
# map at all, which stops the central find_longest_match method
256
# from starting any matching block at a junk element ...
257
# b2j also does not contain entries for "popular" elements, meaning
258
# elements that account for more than 1 + 1% of the total elements, and
259
# when the sequence is reasonably large (>= 200 elements); this can
260
# be viewed as an adaptive notion of semi-junk, and yields an enormous
261
# speedup when, e.g., comparing program files with hundreds of
262
# instances of "return NULL;" ...
263
# note that this is only called when b changes; so for cross-product
264
# kinds of matches, it's best to call set_seq2 once, then set_seq1
265
# repeatedly
266
267
def __chain_b(self):
268
# Because isjunk is a user-defined (not C) function, and we test
269
# for junk a LOT, it's important to minimize the number of calls.
270
# Before the tricks described here, __chain_b was by far the most
271
# time-consuming routine in the whole module! If anyone sees
272
# Jim Roskind, thank him again for profile.py -- I never would
273
# have guessed that.
274
# The first trick is to build b2j ignoring the possibility
275
# of junk. I.e., we don't call isjunk at all yet. Throwing
276
# out the junk later is much cheaper than building b2j "right"
277
# from the start.
278
b = self.b
279
self.b2j = b2j = {}
280
281
36
37
for i, elt in enumerate(b):
282
220
372
indices = b2j.setdefault(elt, [])
283
27
28
indices.append(i)
284
285
# Purge junk elements
286
self.bjunk = junk = set()
287
isjunk = self.isjunk
288
if isjunk:
289
for elt in b2j.keys():
290
if isjunk(elt):
291
junk.add(elt)
292
for elt in junk: # separate loop avoids separate list of keys
293
del b2j[elt]
294
295
# Purge popular elements that are not junk
296
self.bpopular = popular = set()
297
n = len(b)
298
if self.autojunk and n >= 200:
299
ntest = n // 100 + 1
300
29
30
for elt, idxs in b2j.items():
301
32
32
if len(idxs) > ntest:
302
popular.add(elt)
303
for elt in popular: # ditto; as fast for 1% deletion
304
del b2j[elt]
305
306
def find_longest_match(self, alo=0, ahi=None, blo=0, bhi=None):
307
"""Find longest matching block in a[alo:ahi] and b[blo:bhi].
308
309
By default it will find the longest match in the entirety of a and b.
310
311
If isjunk is not defined:
312
313
Return (i,j,k) such that a[i:i+k] is equal to b[j:j+k], where
314
alo <= i <= i+k <= ahi
315
blo <= j <= j+k <= bhi
316
and for all (i',j',k') meeting those conditions,
317
k >= k'
318
i <= i'
319
and if i == i', j <= j'
320
321
In other words, of all maximal matching blocks, return one that
322
starts earliest in a, and of all those maximal matching blocks that
323
start earliest in a, return the one that starts earliest in b.
324
325
>>> s = SequenceMatcher(None, " abcd", "abcd abcd")
326
>>> s.find_longest_match(0, 5, 0, 9)
327
Match(a=0, b=4, size=5)
328
329
If isjunk is defined, first the longest matching block is
330
determined as above, but with the additional restriction that no
331
junk element appears in the block. Then that block is extended as
332
far as possible by matching (only) junk elements on both sides. So
333
the resulting block never matches on junk except as identical junk
334
happens to be adjacent to an "interesting" match.
335
336
Here's the same example as before, but considering blanks to be
337
junk. That prevents " abcd" from matching the " abcd" at the tail
338
end of the second sequence directly. Instead only the "abcd" can
339
match, and matches the leftmost "abcd" in the second sequence:
340
341
>>> s = SequenceMatcher(lambda x: x==" ", " abcd", "abcd abcd")
342
>>> s.find_longest_match(0, 5, 0, 9)
343
Match(a=1, b=0, size=4)
344
345
If no blocks match, return (alo, blo, 0).
346
347
>>> s = SequenceMatcher(None, "ab", "c")
348
>>> s.find_longest_match(0, 2, 0, 1)
349
Match(a=0, b=0, size=0)
350
"""
351
352
# CAUTION: stripping common prefix or suffix would be incorrect.
353
# E.g.,
354
# ab
355
# acab
356
# Longest matching block is "ab", but if common prefix is
357
# stripped, it's "a" (tied with "b"). UNIX(tm) diff does so
358
# strip, so ends up claiming that ab is changed to acab by
359
# inserting "ca" in the middle. That's minimal but unintuitive:
360
# "it's obvious" that someone inserted "ac" at the front.
361
# Windiff ends up at the same place as diff, but by pairing up
362
# the unique 'b's and then matching the first two 'a's.
363
364
2
2
a, b, b2j, isbjunk = self.a, self.b, self.b2j, self.bjunk.__contains__
365
if ahi is None:
366
ahi = len(a)
367
if bhi is None:
368
bhi = len(b)
369
besti, bestj, bestsize = alo, blo, 0
370
# find longest junk-free match
371
# during an iteration of the loop, j2len[j] = length of longest
372
# junk-free match ending with a[i-1] and b[j]
373
j2len = {}
374
nothing = []
375
197
197
for i in range(alo, ahi):
376
# look at all instances of a[i] in b; note that because
377
# b2j has no junk keys, the loop is skipped if a[i] is junk
378
1104
1104
j2lenget = j2len.get
379
225
225
newj2len = {}
380
1916
1916
for j in b2j.get(a[i], nothing):
381
# a[i] matches b[j]
382
399
399
if j < blo:
383
continue
384
113
113
if j >= bhi:
385
break
386
1386
1386
k = newj2len[j] = j2lenget(j-1, 0) + 1
387
334
334
if k > bestsize:
388
75
75
besti, bestj, bestsize = i-k+1, j-k+1, k
389
96
96
j2len = newj2len
390
391
# Extend the best by non-junk elements on each end. In particular,
392
# "popular" non-junk elements aren't in b2j, which greatly speeds
393
# the inner loop above, but also means "the best" match so far
394
# doesn't contain any junk *or* popular non-junk elements.
395
2
2
while besti > alo and bestj > blo and \
396
not isbjunk(b[bestj-1]) and \
397
a[besti-1] == b[bestj-1]:
398
besti, bestj, bestsize = besti-1, bestj-1, bestsize+1
399
while besti+bestsize < ahi and bestj+bestsize < bhi and \
400
1
1
not isbjunk(b[bestj+bestsize]) and \
401
a[besti+bestsize] == b[bestj+bestsize]:
402
bestsize += 1
403
404
# Now that we have a wholly interesting match (albeit possibly
405
# empty!), we may as well suck up the matching junk on each
406
# side of it too. Can't think of a good reason not to, and it
407
# saves post-processing the (possibly considerable) expense of
408
# figuring out what to do with it. In the case of an empty
409
# interesting match, this is clearly the right thing to do,
410
# because no other kind of match is possible in the regions.
411
while besti > alo and bestj > blo and \
412
isbjunk(b[bestj-1]) and \
413
a[besti-1] == b[bestj-1]:
414
besti, bestj, bestsize = besti-1, bestj-1, bestsize+1
415
while besti+bestsize < ahi and bestj+bestsize < bhi and \
416
1
1
isbjunk(b[bestj+bestsize]) and \
417
a[besti+bestsize] == b[bestj+bestsize]:
418
bestsize = bestsize + 1
419
420
3
3
return Match(besti, bestj, bestsize)
421
422
def get_matching_blocks(self):
423
"""Return list of triples describing matching subsequences.
424
425
Each triple is of the form (i, j, n), and means that
426
a[i:i+n] == b[j:j+n]. The triples are monotonically increasing in
427
i and in j. New in Python 2.5, it's also guaranteed that if
428
(i, j, n) and (i', j', n') are adjacent triples in the list, and
429
the second is not the last triple in the list, then i+n != i' or
430
j+n != j'. IOW, adjacent triples never describe adjacent equal
431
blocks.
432
433
The last triple is a dummy, (len(a), len(b), 0), and is the only
434
triple with n==0.
435
436
>>> s = SequenceMatcher(None, "abxcd", "abcd")
437
>>> list(s.get_matching_blocks())
438
[Match(a=0, b=0, size=2), Match(a=3, b=2, size=2), Match(a=5, b=4, size=0)]
439
"""
440
441
if self.matching_blocks is not None:
442
return self.matching_blocks
443
la, lb = len(self.a), len(self.b)
444
445
# This is most naturally expressed as a recursive algorithm, but
446
# at least one user bumped into extreme use cases that exceeded
447
# the recursion limit on their box. So, now we maintain a list
448
# ('queue`) of blocks we still need to look at, and append partial
449
# results to `matching_blocks` in a loop; the matches are sorted
450
# at the end.
451
queue = [(0, la, 0, lb)]
452
matching_blocks = []
453
while queue:
454
alo, ahi, blo, bhi = queue.pop()
455
5
5859
i, j, k = x = self.find_longest_match(alo, ahi, blo, bhi)
456
# a[alo:i] vs b[blo:j] unknown
457
# a[i:i+k] same as b[j:j+k]
458
# a[i+k:ahi] vs b[j+k:bhi] unknown
459
if k: # if k is 0, there was no matching block
460
matching_blocks.append(x)
461
if alo < i and blo < j:
462
queue.append((alo, i, blo, j))
463
if i+k < ahi and j+k < bhi:
464
queue.append((i+k, ahi, j+k, bhi))
465
matching_blocks.sort()
466
467
# It's possible that we have adjacent equal blocks in the
468
# matching_blocks list now. Starting with 2.5, this code was added
469
# to collapse them.
470
i1 = j1 = k1 = 0
471
non_adjacent = []
472
for i2, j2, k2 in matching_blocks:
473
# Is this block adjacent to i1, j1, k1?
474
if i1 + k1 == i2 and j1 + k1 == j2:
475
# Yes, so collapse them -- this just increases the length of
476
# the first block by the length of the second, and the first
477
# block so lengthened remains the block to compare against.
478
k1 += k2
479
else:
480
# Not adjacent. Remember the first block (k1==0 means it's
481
# the dummy we started with), and make the second block the
482
# new block to compare against.
483
if k1:
484
non_adjacent.append((i1, j1, k1))
485
i1, j1, k1 = i2, j2, k2
486
if k1:
487
non_adjacent.append((i1, j1, k1))
488
489
non_adjacent.append( (la, lb, 0) )
490
2
2
self.matching_blocks = list(map(Match._make, non_adjacent))
491
return self.matching_blocks
492
493
def get_opcodes(self):
494
"""Return list of 5-tuples describing how to turn a into b.
495
496
Each tuple is of the form (tag, i1, i2, j1, j2). The first tuple
497
has i1 == j1 == 0, and remaining tuples have i1 == the i2 from the
498
tuple preceding it, and likewise for j1 == the previous j2.
499
500
The tags are strings, with these meanings:
501
502
'replace': a[i1:i2] should be replaced by b[j1:j2]
503
'delete': a[i1:i2] should be deleted.
504
Note that j1==j2 in this case.
505
'insert': b[j1:j2] should be inserted at a[i1:i1].
506
Note that i1==i2 in this case.
507
'equal': a[i1:i2] == b[j1:j2]
508
509
>>> a = "qabxcd"
510
>>> b = "abycdf"
511
>>> s = SequenceMatcher(None, a, b)
512
>>> for tag, i1, i2, j1, j2 in s.get_opcodes():
513
... print(("%7s a[%d:%d] (%s) b[%d:%d] (%s)" %
514
... (tag, i1, i2, a[i1:i2], j1, j2, b[j1:j2])))
515
delete a[0:1] (q) b[0:0] ()
516
equal a[1:3] (ab) b[0:2] (ab)
517
replace a[3:4] (x) b[2:3] (y)
518
equal a[4:6] (cd) b[3:5] (cd)
519
insert a[6:6] () b[5:6] (f)
520
"""
521
522
if self.opcodes is not None:
523
return self.opcodes
524
i = j = 0
525
self.opcodes = answer = []
526
for ai, bj, size in self.get_matching_blocks():
527
# invariant: we've pumped out correct diffs to change
528
# a[:i] into b[:j], and the next matching block is
529
# a[ai:ai+size] == b[bj:bj+size]. So we need to pump
530
# out a diff to change a[i:ai] into b[j:bj], pump out
531
# the matching block, and move (i,j) beyond the match
532
tag = ''
533
if i < ai and j < bj:
534
tag = 'replace'
535
elif i < ai:
536
tag = 'delete'
537
elif j < bj:
538
tag = 'insert'
539
if tag:
540
answer.append( (tag, i, ai, j, bj) )
541
i, j = ai+size, bj+size
542
# the list of matching blocks is terminated by a
543
# sentinel with size 0
544
if size:
545
answer.append( ('equal', ai, i, bj, j) )
546
return answer
547
548
def get_grouped_opcodes(self, n=3):
549
""" Isolate change clusters by eliminating ranges with no changes.
550
551
Return a generator of groups with up to n lines of context.
552
Each group is in the same format as returned by get_opcodes().
553
554
>>> from pprint import pprint
555
>>> a = list(map(str, range(1,40)))
556
>>> b = a[:]
557
>>> b[8:8] = ['i'] # Make an insertion
558
>>> b[20] += 'x' # Make a replacement
559
>>> b[23:28] = [] # Make a deletion
560
>>> b[30] += 'y' # Make another replacement
561
>>> pprint(list(SequenceMatcher(None,a,b).get_grouped_opcodes()))
562
[[('equal', 5, 8, 5, 8), ('insert', 8, 8, 8, 9), ('equal', 8, 11, 9, 12)],
563
[('equal', 16, 19, 17, 20),
564
('replace', 19, 20, 20, 21),
565
('equal', 20, 22, 21, 23),
566
('delete', 22, 27, 23, 23),
567
('equal', 27, 30, 23, 26)],
568
[('equal', 31, 34, 27, 30),
569
('replace', 34, 35, 30, 31),
570
('equal', 35, 38, 31, 34)]]
571
"""
572
573
codes = self.get_opcodes()
574
if not codes:
575
codes = [("equal", 0, 1, 0, 1)]
576
# Fixup leading and trailing groups if they show no changes.
577
if codes[0][0] == 'equal':
578
tag, i1, i2, j1, j2 = codes[0]
579
codes[0] = tag, max(i1, i2-n), i2, max(j1, j2-n), j2
580
if codes[-1][0] == 'equal':
581
tag, i1, i2, j1, j2 = codes[-1]
582
codes[-1] = tag, i1, min(i2, i1+n), j1, min(j2, j1+n)
583
584
nn = n + n
585
group = []
586
for tag, i1, i2, j1, j2 in codes:
587
# End the current group and start a new one whenever
588
# there is a large range with no changes.
589
if tag == 'equal' and i2-i1 > nn:
590
group.append((tag, i1, min(i2, i1+n), j1, min(j2, j1+n)))
591
yield group
592
group = []
593
i1, j1 = max(i1, i2-n), max(j1, j2-n)
594
group.append((tag, i1, i2, j1 ,j2))
595
if group and not (len(group)==1 and group[0][0] == 'equal'):
596
yield group
597
598
def ratio(self):
599
"""Return a measure of the sequences' similarity (float in [0,1]).
600
601
Where T is the total number of elements in both sequences, and
602
M is the number of matches, this is 2.0*M / T.
603
Note that this is 1 if the sequences are identical, and 0 if
604
they have nothing in common.
605
606
.ratio() is expensive to compute if you haven't already computed
607
.get_matching_blocks() or .get_opcodes(), in which case you may
608
want to try .quick_ratio() or .real_quick_ratio() first to get an
609
upper bound.
610
611
>>> s = SequenceMatcher(None, "abcd", "bcde")
612
>>> s.ratio()
613
0.75
614
>>> s.quick_ratio()
615
0.75
616
>>> s.real_quick_ratio()
617
1.0
618
"""
619
620
matches = sum(triple[-1] for triple in self.get_matching_blocks())
621
return _calculate_ratio(matches, len(self.a) + len(self.b))
622
623
def quick_ratio(self):
624
"""Return an upper bound on ratio() relatively quickly.
625
626
This isn't defined beyond that it is an upper bound on .ratio(), and
627
is faster to compute.
628
"""
629
630
# viewing a and b as multisets, set matches to the cardinality
631
# of their intersection; this counts the number of matches
632
# without regard to order, so is clearly an upper bound
633
if self.fullbcount is None:
634
self.fullbcount = fullbcount = {}
635
for elt in self.b:
636
fullbcount[elt] = fullbcount.get(elt, 0) + 1
637
fullbcount = self.fullbcount
638
# avail[x] is the number of times x appears in 'b' less the
639
# number of times we've seen it in 'a' so far ... kinda
640
avail = {}
641
matches = 0
642
for elt in self.a:
643
if elt in avail:
644
numb = avail[elt]
645
else:
646
numb = fullbcount.get(elt, 0)
647
avail[elt] = numb - 1
648
if numb > 0:
649
matches += 1
650
return _calculate_ratio(matches, len(self.a) + len(self.b))
651
652
def real_quick_ratio(self):
653
"""Return an upper bound on ratio() very quickly.
654
655
This isn't defined beyond that it is an upper bound on .ratio(), and
656
is faster to compute than either .ratio() or .quick_ratio().
657
"""
658
659
la, lb = len(self.a), len(self.b)
660
# can't have more matches than the number of elements in the
661
# shorter sequence
662
return _calculate_ratio(min(la, lb), la + lb)
663
664
__class_getitem__ = classmethod(GenericAlias)
665
666
667
def get_close_matches(word, possibilities, n=3, cutoff=0.6):
668
"""Use SequenceMatcher to return list of the best "good enough" matches.
669
670
word is a sequence for which close matches are desired (typically a
671
string).
672
673
possibilities is a list of sequences against which to match word
674
(typically a list of strings).
675
676
Optional arg n (default 3) is the maximum number of close matches to
677
return. n must be > 0.
678
679
Optional arg cutoff (default 0.6) is a float in [0, 1]. Possibilities
680
that don't score at least that similar to word are ignored.
681
682
The best (no more than n) matches among the possibilities are returned
683
in a list, sorted by similarity score, most similar first.
684
685
>>> get_close_matches("appel", ["ape", "apple", "peach", "puppy"])
686
['apple', 'ape']
687
>>> import keyword as _keyword
688
>>> get_close_matches("wheel", _keyword.kwlist)
689
['while']
690
>>> get_close_matches("Apple", _keyword.kwlist)
691
[]
692
>>> get_close_matches("accept", _keyword.kwlist)
693
['except']
694
"""
695
696
if not n > 0:
697
raise ValueError("n must be > 0: %r" % (n,))
698
if not 0.0 <= cutoff <= 1.0:
699
raise ValueError("cutoff must be in [0.0, 1.0]: %r" % (cutoff,))
700
result = []
701
s = SequenceMatcher()
702
s.set_seq2(word)
703
for x in possibilities:
704
s.set_seq1(x)
705
if s.real_quick_ratio() < cutoff or s.quick_ratio() < cutoff:
706
continue
707
708
ratio = s.ratio()
709
if ratio >= cutoff:
710
result.append((ratio, x))
711
712
# Move the best scorers to head of list
713
result = _nlargest(n, result)
714
# Strip scores for the best n matches
715
return [x for score, x in result]
716
717
718
def _keep_original_ws(s, tag_s):
719
"""Replace whitespace with the original whitespace characters in `s`"""
720
return ''.join(
721
c if tag_c == " " and c.isspace() else tag_c
722
for c, tag_c in zip(s, tag_s)
723
)
724
725
726
727
class Differ:
728
r"""
729
Differ is a class for comparing sequences of lines of text, and
730
producing human-readable differences or deltas. Differ uses
731
SequenceMatcher both to compare sequences of lines, and to compare
732
sequences of characters within similar (near-matching) lines.
733
734
Each line of a Differ delta begins with a two-letter code:
735
736
'- ' line unique to sequence 1
737
'+ ' line unique to sequence 2
738
' ' line common to both sequences
739
'? ' line not present in either input sequence
740
741
Lines beginning with '? ' attempt to guide the eye to intraline
742
differences, and were not present in either input sequence. These lines
743
can be confusing if the sequences contain tab characters.
744
745
Note that Differ makes no claim to produce a *minimal* diff. To the
746
contrary, minimal diffs are often counter-intuitive, because they synch
747
up anywhere possible, sometimes accidental matches 100 pages apart.
748
Restricting synch points to contiguous matches preserves some notion of
749
locality, at the occasional cost of producing a longer diff.
750
751
Example: Comparing two texts.
752
753
First we set up the texts, sequences of individual single-line strings
754
ending with newlines (such sequences can also be obtained from the
755
`readlines()` method of file-like objects):
756
757
>>> text1 = ''' 1. Beautiful is better than ugly.
758
... 2. Explicit is better than implicit.
759
... 3. Simple is better than complex.
760
... 4. Complex is better than complicated.
761
... '''.splitlines(keepends=True)
762
>>> len(text1)
763
4
764
>>> text1[0][-1]
765
'\n'
766
>>> text2 = ''' 1. Beautiful is better than ugly.
767
... 3. Simple is better than complex.
768
... 4. Complicated is better than complex.
769
... 5. Flat is better than nested.
770
... '''.splitlines(keepends=True)
771
772
Next we instantiate a Differ object:
773
774
>>> d = Differ()
775
776
Note that when instantiating a Differ object we may pass functions to
777
filter out line and character 'junk'. See Differ.__init__ for details.
778
779
Finally, we compare the two:
780
781
>>> result = list(d.compare(text1, text2))
782
783
'result' is a list of strings, so let's pretty-print it:
784
785
>>> from pprint import pprint as _pprint
786
>>> _pprint(result)
787
[' 1. Beautiful is better than ugly.\n',
788
'- 2. Explicit is better than implicit.\n',
789
'- 3. Simple is better than complex.\n',
790
'+ 3. Simple is better than complex.\n',
791
'? ++\n',
792
'- 4. Complex is better than complicated.\n',
793
'? ^ ---- ^\n',
794
'+ 4. Complicated is better than complex.\n',
795
'? ++++ ^ ^\n',
796
'+ 5. Flat is better than nested.\n']
797
798
As a single multi-line string it looks like this:
799
800
>>> print(''.join(result), end="")
801
1. Beautiful is better than ugly.
802
- 2. Explicit is better than implicit.
803
- 3. Simple is better than complex.
804
+ 3. Simple is better than complex.
805
? ++
806
- 4. Complex is better than complicated.
807
? ^ ---- ^
808
+ 4. Complicated is better than complex.
809
? ++++ ^ ^
810
+ 5. Flat is better than nested.
811
"""
812
813
def __init__(self, linejunk=None, charjunk=None):
814
"""
815
Construct a text differencer, with optional filters.
816
817
The two optional keyword parameters are for filter functions:
818
819
- `linejunk`: A function that should accept a single string argument,
820
and return true iff the string is junk. The module-level function
821
`IS_LINE_JUNK` may be used to filter out lines without visible
822
characters, except for at most one splat ('#'). It is recommended
823
to leave linejunk None; the underlying SequenceMatcher class has
824
an adaptive notion of "noise" lines that's better than any static
825
definition the author has ever been able to craft.
826
827
- `charjunk`: A function that should accept a string of length 1. The
828
module-level function `IS_CHARACTER_JUNK` may be used to filter out
829
whitespace characters (a blank or tab; **note**: bad idea to include
830
newline in this!). Use of IS_CHARACTER_JUNK is recommended.
831
"""
832
833
self.linejunk = linejunk
834
self.charjunk = charjunk
835
836
def compare(self, a, b):
837
r"""
838
Compare two sequences of lines; generate the resulting delta.
839
840
Each sequence must contain individual single-line strings ending with
841
newlines. Such sequences can be obtained from the `readlines()` method
842
of file-like objects. The delta generated also consists of newline-
843
terminated strings, ready to be printed as-is via the writelines()
844
method of a file-like object.
845
846
Example:
847
848
>>> print(''.join(Differ().compare('one\ntwo\nthree\n'.splitlines(True),
849
... 'ore\ntree\nemu\n'.splitlines(True))),
850
... end="")
851
- one
852
? ^
853
+ ore
854
? ^
855
- two
856
- three
857
? -
858
+ tree
859
+ emu
860
"""
861
862
cruncher = SequenceMatcher(self.linejunk, a, b)
863
for tag, alo, ahi, blo, bhi in cruncher.get_opcodes():
864
if tag == 'replace':
865
g = self._fancy_replace(a, alo, ahi, b, blo, bhi)
866
elif tag == 'delete':
867
g = self._dump('-', a, alo, ahi)
868
elif tag == 'insert':
869
g = self._dump('+', b, blo, bhi)
870
elif tag == 'equal':
871
g = self._dump(' ', a, alo, ahi)
872
else:
873
raise ValueError('unknown tag %r' % (tag,))
874
875
yield from g
876
877
def _dump(self, tag, x, lo, hi):
878
"""Generate comparison results for a same-tagged range."""
879
for i in range(lo, hi):
880
yield '%s %s' % (tag, x[i])
881
882
def _plain_replace(self, a, alo, ahi, b, blo, bhi):
883
assert alo < ahi and blo < bhi
884
# dump the shorter block first -- reduces the burden on short-term
885
# memory if the blocks are of very different sizes
886
if bhi - blo < ahi - alo:
887
first = self._dump('+', b, blo, bhi)
888
second = self._dump('-', a, alo, ahi)
889
else:
890
first = self._dump('-', a, alo, ahi)
891
second = self._dump('+', b, blo, bhi)
892
893
for g in first, second:
894
yield from g
895
896
def _fancy_replace(self, a, alo, ahi, b, blo, bhi):
897
r"""
898
When replacing one block of lines with another, search the blocks
899
for *similar* lines; the best-matching pair (if any) is used as a
900
synch point, and intraline difference marking is done on the
901
similar pair. Lots of work, but often worth it.
902
903
Example:
904
905
>>> d = Differ()
906
>>> results = d._fancy_replace(['abcDefghiJkl\n'], 0, 1,
907
... ['abcdefGhijkl\n'], 0, 1)
908
>>> print(''.join(results), end="")
909
- abcDefghiJkl
910
? ^ ^ ^
911
+ abcdefGhijkl
912
? ^ ^ ^
913
"""
914
# Don't synch up unless the lines have a similarity score above
915
# cutoff. Previously only the smallest pair was handled here,
916
# and if there are many pairs with the best ratio, recursion
917
# could grow very deep, and runtime cubic. See:
918
# https://github.com/python/cpython/issues/119105
919
#
920
# Later, more pathological cases prompted removing recursion
921
# entirely.
922
cutoff = 0.74999
923
cruncher = SequenceMatcher(self.charjunk)
924
crqr = cruncher.real_quick_ratio
925
cqr = cruncher.quick_ratio
926
cr = cruncher.ratio
927
928
WINDOW = 10
929
best_i = best_j = None
930
dump_i, dump_j = alo, blo # smallest indices not yet resolved
931
for j in range(blo, bhi):
932
cruncher.set_seq2(b[j])
933
# Search the corresponding i's within WINDOW for rhe highest
934
# ratio greater than `cutoff`.
935
aequiv = alo + (j - blo)
936
arange = range(max(aequiv - WINDOW, dump_i),
937
min(aequiv + WINDOW + 1, ahi))
938
if not arange: # likely exit if `a` is shorter than `b`
939
break
940
best_ratio = cutoff
941
for i in arange:
942
cruncher.set_seq1(a[i])
943
# Ordering by cheapest to most expensive ratio is very
944
# valuable, most often getting out early.
945
if crqr() <= best_ratio or cqr() <= best_ratio:
946
continue
947
948
ratio = cr()
949
if ratio > best_ratio:
950
best_i, best_j, best_ratio = i, j, ratio
951
952
if best_i is None:
953
# found nothing to synch on yet - move to next j
954
continue
955
956
# pump out straight replace from before this synch pair
957
yield from self._fancy_helper(a, dump_i, best_i,
958
b, dump_j, best_j)
959
# do intraline marking on the synch pair
960
aelt, belt = a[best_i], b[best_j]
961
if aelt != belt:
962
# pump out a '-', '?', '+', '?' quad for the synched lines
963
atags = btags = ""
964
cruncher.set_seqs(aelt, belt)
965
for tag, ai1, ai2, bj1, bj2 in cruncher.get_opcodes():
966
la, lb = ai2 - ai1, bj2 - bj1
967
if tag == 'replace':
968
atags += '^' * la
969
btags += '^' * lb
970
elif tag == 'delete':
971
atags += '-' * la
972
elif tag == 'insert':
973
btags += '+' * lb
974
elif tag == 'equal':
975
atags += ' ' * la
976
btags += ' ' * lb
977
else:
978
raise ValueError('unknown tag %r' % (tag,))
979
yield from self._qformat(aelt, belt, atags, btags)
980
else:
981
# the synch pair is identical
982
yield ' ' + aelt
983
dump_i, dump_j = best_i + 1, best_j + 1
984
best_i = best_j = None
985
986
# pump out straight replace from after the last synch pair
987
yield from self._fancy_helper(a, dump_i, ahi,
988
b, dump_j, bhi)
989
990
def _fancy_helper(self, a, alo, ahi, b, blo, bhi):
991
g = []
992
if alo < ahi:
993
if blo < bhi:
994
g = self._plain_replace(a, alo, ahi, b, blo, bhi)
995
else:
996
g = self._dump('-', a, alo, ahi)
997
elif blo < bhi:
998
g = self._dump('+', b, blo, bhi)
999
1000
yield from g
1001
1002
def _qformat(self, aline, bline, atags, btags):
1003
r"""
1004
Format "?" output and deal with tabs.
1005
1006
Example:
1007
1008
>>> d = Differ()
1009
>>> results = d._qformat('\tabcDefghiJkl\n', '\tabcdefGhijkl\n',
1010
... ' ^ ^ ^ ', ' ^ ^ ^ ')
1011
>>> for line in results: print(repr(line))
1012
...
1013
'- \tabcDefghiJkl\n'
1014
'? \t ^ ^ ^\n'
1015
'+ \tabcdefGhijkl\n'
1016
'? \t ^ ^ ^\n'
1017
"""
1018
atags = _keep_original_ws(aline, atags).rstrip()
1019
btags = _keep_original_ws(bline, btags).rstrip()
1020
1021
yield "- " + aline
1022
if atags:
1023
yield f"? {atags}\n"
1024
1025
yield "+ " + bline
1026
if btags:
1027
yield f"? {btags}\n"
1028
1029
# With respect to junk, an earlier version of ndiff simply refused to
1030
# *start* a match with a junk element. The result was cases like this:
1031
# before: private Thread currentThread;
1032
# after: private volatile Thread currentThread;
1033
# If you consider whitespace to be junk, the longest contiguous match
1034
# not starting with junk is "e Thread currentThread". So ndiff reported
1035
# that "e volatil" was inserted between the 't' and the 'e' in "private".
1036
# While an accurate view, to people that's absurd. The current version
1037
# looks for matching blocks that are entirely junk-free, then extends the
1038
# longest one of those as far as possible but only with matching junk.
1039
# So now "currentThread" is matched, then extended to suck up the
1040
# preceding blank; then "private" is matched, and extended to suck up the
1041
# following blank; then "Thread" is matched; and finally ndiff reports
1042
# that "volatile " was inserted before "Thread". The only quibble
1043
# remaining is that perhaps it was really the case that " volatile"
1044
# was inserted after "private". I can live with that <wink>.
1045
1046
def IS_LINE_JUNK(line, pat=None):
1047
r"""
1048
Return True for ignorable line: if `line` is blank or contains a single '#'.
1049
1050
Examples:
1051
1052
>>> IS_LINE_JUNK('\n')
1053
True
1054
>>> IS_LINE_JUNK(' # \n')
1055
True
1056
>>> IS_LINE_JUNK('hello\n')
1057
False
1058
"""
1059
1060
if pat is None:
1061
# Default: match '#' or the empty string
1062
return line.strip() in '#'
1063
# Previous versions used the undocumented parameter 'pat' as a
1064
# match function. Retain this behaviour for compatibility.
1065
return pat(line) is not None
1066
1067
def IS_CHARACTER_JUNK(ch, ws=" \t"):
1068
r"""
1069
Return True for ignorable character: iff `ch` is a space or tab.
1070
1071
Examples:
1072
1073
>>> IS_CHARACTER_JUNK(' ')
1074
True
1075
>>> IS_CHARACTER_JUNK('\t')
1076
True
1077
>>> IS_CHARACTER_JUNK('\n')
1078
False
1079
>>> IS_CHARACTER_JUNK('x')
1080
False
1081
"""
1082
1083
return ch in ws
1084
1085
1086
########################################################################
1087
### Unified Diff
1088
########################################################################
1089
1090
def _format_range_unified(start, stop):
1091
'Convert range to the "ed" format'
1092
# Per the diff spec at http://www.unix.org/single_unix_specification/
1093
beginning = start + 1 # lines start numbering with one
1094
length = stop - start
1095
if length == 1:
1096
return '{}'.format(beginning)
1097
if not length:
1098
beginning -= 1 # empty ranges begin at line just before the range
1099
return '{},{}'.format(beginning, length)
1100
1101
def unified_diff(a, b, fromfile='', tofile='', fromfiledate='',
1102
tofiledate='', n=3, lineterm='\n', *, color=False):
1103
r"""
1104
Compare two sequences of lines; generate the delta as a unified diff.
1105
1106
Unified diffs are a compact way of showing line changes and a few
1107
lines of context. The number of context lines is set by 'n' which
1108
defaults to three.
1109
1110
By default, the diff control lines (those with ---, +++, or @@) are
1111
created with a trailing newline. This is helpful so that inputs
1112
created from file.readlines() result in diffs that are suitable for
1113
file.writelines() since both the inputs and outputs have trailing
1114
newlines.
1115
1116
For inputs that do not have trailing newlines, set the lineterm
1117
argument to "" so that the output will be uniformly newline free.
1118
1119
Set 'color' to True to enable output in color, similar to
1120
'git diff --color'. Even if enabled, it can be
1121
controlled using environment variables such as 'NO_COLOR'.
1122
1123
The unidiff format normally has a header for filenames and modification
1124
times. Any or all of these may be specified using strings for
1125
'fromfile', 'tofile', 'fromfiledate', and 'tofiledate'.
1126
The modification times are normally expressed in the ISO 8601 format.
1127
1128
Example:
1129
1130
>>> for line in unified_diff('one two three four'.split(),
1131
... 'zero one tree four'.split(), 'Original', 'Current',
1132
... '2005-01-26 23:30:50', '2010-04-02 10:20:52',
1133
... lineterm=''):
1134
... print(line) # doctest: +NORMALIZE_WHITESPACE
1135
--- Original 2005-01-26 23:30:50
1136
+++ Current 2010-04-02 10:20:52
1137
@@ -1,4 +1,4 @@
1138
+zero
1139
one
1140
-two
1141
-three
1142
+tree
1143
four
1144
"""
1145
1146
if color and can_colorize():
1147
t = get_theme(force_color=True).difflib
1148
else:
1149
t = get_theme(force_no_color=True).difflib
1150
1151
_check_types(a, b, fromfile, tofile, fromfiledate, tofiledate, lineterm)
1152
started = False
1153
for group in SequenceMatcher(None,a,b).get_grouped_opcodes(n):
1154
if not started:
1155
started = True
1156
fromdate = '\t{}'.format(fromfiledate) if fromfiledate else ''
1157
todate = '\t{}'.format(tofiledate) if tofiledate else ''
1158
yield f'{t.header}--- {fromfile}{fromdate}{lineterm}{t.reset}'
1159
yield f'{t.header}+++ {tofile}{todate}{lineterm}{t.reset}'
1160
1161
first, last = group[0], group[-1]
1162
file1_range = _format_range_unified(first[1], last[2])
1163
file2_range = _format_range_unified(first[3], last[4])
1164
yield f'{t.hunk}@@ -{file1_range} +{file2_range} @@{lineterm}{t.reset}'
1165
1166
for tag, i1, i2, j1, j2 in group:
1167
if tag == 'equal':
1168
for line in a[i1:i2]:
1169
yield f'{t.context} {line}{t.reset}'
1170
continue
1171
if tag in {'replace', 'delete'}:
1172
for line in a[i1:i2]:
1173
yield f'{t.removed}-{line}{t.reset}'
1174
if tag in {'replace', 'insert'}:
1175
for line in b[j1:j2]:
1176
yield f'{t.added}+{line}{t.reset}'
1177
1178
1179
########################################################################
1180
### Context Diff
1181
########################################################################
1182
1183
def _format_range_context(start, stop):
1184
'Convert range to the "ed" format'
1185
# Per the diff spec at http://www.unix.org/single_unix_specification/
1186
beginning = start + 1 # lines start numbering with one
1187
length = stop - start
1188
if not length:
1189
beginning -= 1 # empty ranges begin at line just before the range
1190
if length <= 1:
1191
return '{}'.format(beginning)
1192
return '{},{}'.format(beginning, beginning + length - 1)
1193
1194
# See http://www.unix.org/single_unix_specification/
1195
def context_diff(a, b, fromfile='', tofile='',
1196
fromfiledate='', tofiledate='', n=3, lineterm='\n'):
1197
r"""
1198
Compare two sequences of lines; generate the delta as a context diff.
1199
1200
Context diffs are a compact way of showing line changes and a few
1201
lines of context. The number of context lines is set by 'n' which
1202
defaults to three.
1203
1204
By default, the diff control lines (those with *** or ---) are
1205
created with a trailing newline. This is helpful so that inputs
1206
created from file.readlines() result in diffs that are suitable for
1207
file.writelines() since both the inputs and outputs have trailing
1208
newlines.
1209
1210
For inputs that do not have trailing newlines, set the lineterm
1211
argument to "" so that the output will be uniformly newline free.
1212
1213
The context diff format normally has a header for filenames and
1214
modification times. Any or all of these may be specified using
1215
strings for 'fromfile', 'tofile', 'fromfiledate', and 'tofiledate'.
1216
The modification times are normally expressed in the ISO 8601 format.
1217
If not specified, the strings default to blanks.
1218
1219
Example:
1220
1221
>>> print(''.join(context_diff('one\ntwo\nthree\nfour\n'.splitlines(True),
1222
... 'zero\none\ntree\nfour\n'.splitlines(True), 'Original', 'Current')),
1223
... end="")
1224
*** Original
1225
--- Current
1226
***************
1227
*** 1,4 ****
1228
one
1229
! two
1230
! three
1231
four
1232
--- 1,4 ----
1233
+ zero
1234
one
1235
! tree
1236
four
1237
"""
1238
1239
_check_types(a, b, fromfile, tofile, fromfiledate, tofiledate, lineterm)
1240
prefix = dict(insert='+ ', delete='- ', replace='! ', equal=' ')
1241
started = False
1242
for group in SequenceMatcher(None,a,b).get_grouped_opcodes(n):
1243
if not started:
1244
started = True
1245
fromdate = '\t{}'.format(fromfiledate) if fromfiledate else ''
1246
todate = '\t{}'.format(tofiledate) if tofiledate else ''
1247
yield '*** {}{}{}'.format(fromfile, fromdate, lineterm)
1248
yield '--- {}{}{}'.format(tofile, todate, lineterm)
1249
1250
first, last = group[0], group[-1]
1251
yield '***************' + lineterm
1252
1253
file1_range = _format_range_context(first[1], last[2])
1254
yield '*** {} ****{}'.format(file1_range, lineterm)
1255
1256
if any(tag in {'replace', 'delete'} for tag, _, _, _, _ in group):
1257
for tag, i1, i2, _, _ in group:
1258
if tag != 'insert':
1259
for line in a[i1:i2]:
1260
yield prefix[tag] + line
1261
1262
file2_range = _format_range_context(first[3], last[4])
1263
yield '--- {} ----{}'.format(file2_range, lineterm)
1264
1265
if any(tag in {'replace', 'insert'} for tag, _, _, _, _ in group):
1266
for tag, _, _, j1, j2 in group:
1267
if tag != 'delete':
1268
for line in b[j1:j2]:
1269
yield prefix[tag] + line
1270
1271
def _check_types(a, b, *args):
1272
# Checking types is weird, but the alternative is garbled output when
1273
# someone passes mixed bytes and str to {unified,context}_diff(). E.g.
1274
# without this check, passing filenames as bytes results in output like
1275
# --- b'oldfile.txt'
1276
# +++ b'newfile.txt'
1277
# because of how str.format() incorporates bytes objects.
1278
if a and not isinstance(a[0], str):
1279
raise TypeError('lines to compare must be str, not %s (%r)' %
1280
(type(a[0]).__name__, a[0]))
1281
if b and not isinstance(b[0], str):
1282
raise TypeError('lines to compare must be str, not %s (%r)' %
1283
(type(b[0]).__name__, b[0]))
1284
if isinstance(a, str):
1285
raise TypeError('input must be a sequence of strings, not %s' %
1286
type(a).__name__)
1287
if isinstance(b, str):
1288
raise TypeError('input must be a sequence of strings, not %s' %
1289
type(b).__name__)
1290
for arg in args:
1291
if not isinstance(arg, str):
1292
raise TypeError('all arguments must be str, not: %r' % (arg,))
1293
1294
def diff_bytes(dfunc, a, b, fromfile=b'', tofile=b'',
1295
fromfiledate=b'', tofiledate=b'', n=3, lineterm=b'\n'):
1296
r"""
1297
Compare `a` and `b`, two sequences of lines represented as bytes rather
1298
than str. This is a wrapper for `dfunc`, which is typically either
1299
unified_diff() or context_diff(). Inputs are losslessly converted to
1300
strings so that `dfunc` only has to worry about strings, and encoded
1301
back to bytes on return. This is necessary to compare files with
1302
unknown or inconsistent encoding. All other inputs (except `n`) must be
1303
bytes rather than str.
1304
"""
1305
def decode(s):
1306
try:
1307
return s.decode('ascii', 'surrogateescape')
1308
except AttributeError as err:
1309
msg = ('all arguments must be bytes, not %s (%r)' %
1310
(type(s).__name__, s))
1311
raise TypeError(msg) from err
1312
a = list(map(decode, a))
1313
b = list(map(decode, b))
1314
fromfile = decode(fromfile)
1315
tofile = decode(tofile)
1316
fromfiledate = decode(fromfiledate)
1317
tofiledate = decode(tofiledate)
1318
lineterm = decode(lineterm)
1319
1320
lines = dfunc(a, b, fromfile, tofile, fromfiledate, tofiledate, n, lineterm)
1321
for line in lines:
1322
yield line.encode('ascii', 'surrogateescape')
1323
1324
def ndiff(a, b, linejunk=None, charjunk=IS_CHARACTER_JUNK):
1325
r"""
1326
Compare `a` and `b` (lists of strings); return a `Differ`-style delta.
1327
1328
Optional keyword parameters `linejunk` and `charjunk` are for filter
1329
functions, or can be None:
1330
1331
- linejunk: A function that should accept a single string argument and
1332
return true iff the string is junk. The default is None, and is
1333
recommended; the underlying SequenceMatcher class has an adaptive
1334
notion of "noise" lines.
1335
1336
- charjunk: A function that accepts a character (string of length
1337
1), and returns true iff the character is junk. The default is
1338
the module-level function IS_CHARACTER_JUNK, which filters out
1339
whitespace characters (a blank or tab; note: it's a bad idea to
1340
include newline in this!).
1341
1342
Tools/scripts/ndiff.py is a command-line front-end to this function.
1343
1344
Example:
1345
1346
>>> diff = ndiff('one\ntwo\nthree\n'.splitlines(keepends=True),
1347
... 'ore\ntree\nemu\n'.splitlines(keepends=True))
1348
>>> print(''.join(diff), end="")
1349
- one
1350
? ^
1351
+ ore
1352
? ^
1353
- two
1354
- three
1355
? -
1356
+ tree
1357
+ emu
1358
"""
1359
return Differ(linejunk, charjunk).compare(a, b)
1360
1361
def _mdiff(fromlines, tolines, context=None, linejunk=None,
1362
charjunk=IS_CHARACTER_JUNK):
1363
r"""Returns generator yielding marked up from/to side by side differences.
1364
1365
Arguments:
1366
fromlines -- list of text lines to compared to tolines
1367
tolines -- list of text lines to be compared to fromlines
1368
context -- number of context lines to display on each side of difference,
1369
if None, all from/to text lines will be generated.
1370
linejunk -- passed on to ndiff (see ndiff documentation)
1371
charjunk -- passed on to ndiff (see ndiff documentation)
1372
1373
This function returns an iterator which returns a tuple:
1374
(from line tuple, to line tuple, boolean flag)
1375
1376
from/to line tuple -- (line num, line text)
1377
line num -- integer or None (to indicate a context separation)
1378
line text -- original line text with following markers inserted:
1379
'\0+' -- marks start of added text
1380
'\0-' -- marks start of deleted text
1381
'\0^' -- marks start of changed text
1382
'\1' -- marks end of added/deleted/changed text
1383
1384
boolean flag -- None indicates context separation, True indicates
1385
either "from" or "to" line contains a change, otherwise False.
1386
1387
This function/iterator was originally developed to generate side by side
1388
file difference for making HTML pages (see HtmlDiff class for example
1389
usage).
1390
1391
Note, this function utilizes the ndiff function to generate the side by
1392
side difference markup. Optional ndiff arguments may be passed to this
1393
function and they in turn will be passed to ndiff.
1394
"""
1395
import re
1396
1397
# regular expression for finding intraline change indices
1398
change_re = re.compile(r'(\++|\-+|\^+)')
1399
1400
# create the difference iterator to generate the differences
1401
diff_lines_iterator = ndiff(fromlines,tolines,linejunk,charjunk)
1402
1403
def _make_line(lines, format_key, side, num_lines=[0,0]):
1404
"""Returns line of text with user's change markup and line formatting.
1405
1406
lines -- list of lines from the ndiff generator to produce a line of
1407
text from. When producing the line of text to return, the
1408
lines used are removed from this list.
1409
format_key -- '+' return first line in list with "add" markup around
1410
the entire line.
1411
'-' return first line in list with "delete" markup around
1412
the entire line.
1413
'?' return first line in list with add/delete/change
1414
intraline markup (indices obtained from second line)
1415
None return first line in list with no markup
1416
side -- indice into the num_lines list (0=from,1=to)
1417
num_lines -- from/to current line number. This is NOT intended to be a
1418
passed parameter. It is present as a keyword argument to
1419
maintain memory of the current line numbers between calls
1420
of this function.
1421
1422
Note, this function is purposefully not defined at the module scope so
1423
that data it needs from its parent function (within whose context it
1424
is defined) does not need to be of module scope.
1425
"""
1426
num_lines[side] += 1
1427
# Handle case where no user markup is to be added, just return line of
1428
# text with user's line format to allow for usage of the line number.
1429
if format_key is None:
1430
return (num_lines[side],lines.pop(0)[2:])
1431
# Handle case of intraline changes
1432
if format_key == '?':
1433
text, markers = lines.pop(0), lines.pop(0)
1434
# find intraline changes (store change type and indices in tuples)
1435
sub_info = []
1436
def record_sub_info(match_object,sub_info=sub_info):
1437
sub_info.append([match_object.group(1)[0],match_object.span()])
1438
return match_object.group(1)
1439
change_re.sub(record_sub_info,markers)
1440
# process each tuple inserting our special marks that won't be
1441
# noticed by an xml/html escaper.
1442
for key,(begin,end) in reversed(sub_info):
1443
text = text[0:begin]+'\0'+key+text[begin:end]+'\1'+text[end:]
1444
text = text[2:]
1445
# Handle case of add/delete entire line
1446
else:
1447
text = lines.pop(0)[2:]
1448
# if line of text is just a newline, insert a space so there is
1449
# something for the user to highlight and see.
1450
if not text:
1451
text = ' '
1452
# insert marks that won't be noticed by an xml/html escaper.
1453
text = '\0' + format_key + text + '\1'
1454
# Return line of text, first allow user's line formatter to do its
1455
# thing (such as adding the line number) then replace the special
1456
# marks with what the user's change markup.
1457
return (num_lines[side],text)
1458
1459
def _line_iterator():
1460
"""Yields from/to lines of text with a change indication.
1461
1462
This function is an iterator. It itself pulls lines from a
1463
differencing iterator, processes them and yields them. When it can
1464
it yields both a "from" and a "to" line, otherwise it will yield one
1465
or the other. In addition to yielding the lines of from/to text, a
1466
boolean flag is yielded to indicate if the text line(s) have
1467
differences in them.
1468
1469
Note, this function is purposefully not defined at the module scope so
1470
that data it needs from its parent function (within whose context it
1471
is defined) does not need to be of module scope.
1472
"""
1473
lines = []
1474
num_blanks_pending, num_blanks_to_yield = 0, 0
1475
while True:
1476
# Load up next 4 lines so we can look ahead, create strings which
1477
# are a concatenation of the first character of each of the 4 lines
1478
# so we can do some very readable comparisons.
1479
while len(lines) < 4:
1480
lines.append(next(diff_lines_iterator, 'X'))
1481
s = ''.join([line[0] for line in lines])
1482
if s.startswith('X'):
1483
# When no more lines, pump out any remaining blank lines so the
1484
# corresponding add/delete lines get a matching blank line so
1485
# all line pairs get yielded at the next level.
1486
num_blanks_to_yield = num_blanks_pending
1487
elif s.startswith('-?+?'):
1488
# simple intraline change
1489
yield _make_line(lines,'?',0), _make_line(lines,'?',1), True
1490
continue
1491
elif s.startswith('--++'):
1492
# in delete block, add block coming: we do NOT want to get
1493
# caught up on blank lines yet, just process the delete line
1494
num_blanks_pending -= 1
1495
yield _make_line(lines,'-',0), None, True
1496
continue
1497
elif s.startswith(('--?+', '--+', '- ')):
1498
# in delete block and see an intraline change or unchanged line
1499
# coming: yield the delete line and then blanks
1500
from_line,to_line = _make_line(lines,'-',0), None
1501
num_blanks_to_yield,num_blanks_pending = num_blanks_pending-1,0
1502
elif s.startswith('-+?'):
1503
# intraline change
1504
yield _make_line(lines,None,0), _make_line(lines,'?',1), True
1505
continue
1506
elif s.startswith('-?+'):
1507
# intraline change
1508
yield _make_line(lines,'?',0), _make_line(lines,None,1), True
1509
continue
1510
elif s.startswith('-'):
1511
# delete FROM line
1512
num_blanks_pending -= 1
1513
yield _make_line(lines,'-',0), None, True
1514
continue
1515
elif s.startswith('+--'):
1516
# in add block, delete block coming: we do NOT want to get
1517
# caught up on blank lines yet, just process the add line
1518
num_blanks_pending += 1
1519
yield None, _make_line(lines,'+',1), True
1520
continue
1521
elif s.startswith(('+ ', '+-')):
1522
# will be leaving an add block: yield blanks then add line
1523
from_line, to_line = None, _make_line(lines,'+',1)
1524
num_blanks_to_yield,num_blanks_pending = num_blanks_pending+1,0
1525
elif s.startswith('+'):
1526
# inside an add block, yield the add line
1527
num_blanks_pending += 1
1528
yield None, _make_line(lines,'+',1), True
1529
continue
1530
elif s.startswith(' '):
1531
# unchanged text, yield it to both sides
1532
yield _make_line(lines[:],None,0),_make_line(lines,None,1),False
1533
continue
1534
# Catch up on the blank lines so when we yield the next from/to
1535
# pair, they are lined up.
1536
while(num_blanks_to_yield < 0):
1537
num_blanks_to_yield += 1
1538
yield None,('','\n'),True
1539
while(num_blanks_to_yield > 0):
1540
num_blanks_to_yield -= 1
1541
yield ('','\n'),None,True
1542
if s.startswith('X'):
1543
return
1544
else:
1545
yield from_line,to_line,True
1546
1547
def _line_pair_iterator():
1548
"""Yields from/to lines of text with a change indication.
1549
1550
This function is an iterator. It itself pulls lines from the line
1551
iterator. Its difference from that iterator is that this function
1552
always yields a pair of from/to text lines (with the change
1553
indication). If necessary it will collect single from/to lines
1554
until it has a matching pair from/to pair to yield.
1555
1556
Note, this function is purposefully not defined at the module scope so
1557
that data it needs from its parent function (within whose context it
1558
is defined) does not need to be of module scope.
1559
"""
1560
line_iterator = _line_iterator()
1561
fromlines,tolines=[],[]
1562
while True:
1563
# Collecting lines of text until we have a from/to pair
1564
while (len(fromlines)==0 or len(tolines)==0):
1565
try:
1566
from_line, to_line, found_diff = next(line_iterator)
1567
except StopIteration:
1568
return
1569
if from_line is not None:
1570
fromlines.append((from_line,found_diff))
1571
if to_line is not None:
1572
tolines.append((to_line,found_diff))
1573
# Once we have a pair, remove them from the collection and yield it
1574
from_line, fromDiff = fromlines.pop(0)
1575
to_line, to_diff = tolines.pop(0)
1576
yield (from_line,to_line,fromDiff or to_diff)
1577
1578
# Handle case where user does not want context differencing, just yield
1579
# them up without doing anything else with them.
1580
line_pair_iterator = _line_pair_iterator()
1581
if context is None:
1582
yield from line_pair_iterator
1583
# Handle case where user wants context differencing. We must do some
1584
# storage of lines until we know for sure that they are to be yielded.
1585
else:
1586
context += 1
1587
lines_to_write = 0
1588
while True:
1589
# Store lines up until we find a difference, note use of a
1590
# circular queue because we only need to keep around what
1591
# we need for context.
1592
index, contextLines = 0, [None]*(context)
1593
found_diff = False
1594
while(found_diff is False):
1595
try:
1596
from_line, to_line, found_diff = next(line_pair_iterator)
1597
except StopIteration:
1598
return
1599
i = index % context
1600
contextLines[i] = (from_line, to_line, found_diff)
1601
index += 1
1602
# Yield lines that we have collected so far, but first yield
1603
# the user's separator.
1604
if index > context:
1605
yield None, None, None
1606
lines_to_write = context
1607
else:
1608
lines_to_write = index
1609
index = 0
1610
while(lines_to_write):
1611
i = index % context
1612
index += 1
1613
yield contextLines[i]
1614
lines_to_write -= 1
1615
# Now yield the context lines after the change
1616
lines_to_write = context-1
1617
try:
1618
while(lines_to_write):
1619
from_line, to_line, found_diff = next(line_pair_iterator)
1620
# If another change within the context, extend the context
1621
if found_diff:
1622
lines_to_write = context-1
1623
else:
1624
lines_to_write -= 1
1625
yield from_line, to_line, found_diff
1626
except StopIteration:
1627
# Catch exception from next() and return normally
1628
return
1629
1630
1631
_file_template = """
1632
<!DOCTYPE html>
1633
<html lang="en">
1634
<head>
1635
<meta charset="%(charset)s">
1636
<meta name="viewport" content="width=device-width, initial-scale=1">
1637
<title>Diff comparison</title>
1638
<style>%(styles)s
1639
</style>
1640
</head>
1641
1642
<body>
1643
%(table)s%(legend)s
1644
</body>
1645
1646
</html>"""
1647
1648
_styles = """
1649
:root {color-scheme: light dark}
1650
table.diff {
1651
font-family: Menlo, Consolas, Monaco, Liberation Mono, Lucida Console, monospace;
1652
border: medium;
1653
}
1654
.diff_header {
1655
background-color: #e0e0e0;
1656
font-weight: bold;
1657
}
1658
td.diff_header {
1659
text-align: right;
1660
padding: 0 8px;
1661
}
1662
.diff_next {
1663
background-color: #c0c0c0;
1664
padding: 4px 0;
1665
}
1666
.diff_add {background-color:palegreen}
1667
.diff_chg {background-color:#ffff77}
1668
.diff_sub {background-color:#ffaaaa}
1669
table.diff[summary="Legends"] {
1670
margin-top: 20px;
1671
border: 1px solid #ccc;
1672
}
1673
table.diff[summary="Legends"] th {
1674
background-color: #e0e0e0;
1675
padding: 4px 8px;
1676
}
1677
table.diff[summary="Legends"] td {
1678
padding: 4px 8px;
1679
}
1680
1681
@media (prefers-color-scheme: dark) {
1682
.diff_header {background-color:#666}
1683
.diff_next {background-color:#393939}
1684
.diff_add {background-color:darkgreen}
1685
.diff_chg {background-color:#847415}
1686
.diff_sub {background-color:darkred}
1687
table.diff[summary="Legends"] {border-color:#555}
1688
table.diff[summary="Legends"] th{background-color:#666}
1689
}"""
1690
1691
_table_template = """
1692
<table class="diff" id="difflib_chg_%(prefix)s_top"
1693
cellspacing="0" cellpadding="0" rules="groups" >
1694
<colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>
1695
<colgroup></colgroup> <colgroup></colgroup> <colgroup></colgroup>
1696
%(header_row)s
1697
<tbody>
1698
%(data_rows)s </tbody>
1699
</table>"""
1700
1701
_legend = """
1702
<table class="diff" summary="Legends">
1703
<tr> <th colspan="2"> Legends </th> </tr>
1704
<tr> <td> <table border="" summary="Colors">
1705
<tr><th> Colors </th> </tr>
1706
<tr><td class="diff_add">&nbsp;Added&nbsp;</td></tr>
1707
<tr><td class="diff_chg">Changed</td> </tr>
1708
<tr><td class="diff_sub">Deleted</td> </tr>
1709
</table></td>
1710
<td> <table border="" summary="Links">
1711
<tr><th colspan="2"> Links </th> </tr>
1712
<tr><td>(f)irst change</td> </tr>
1713
<tr><td>(n)ext change</td> </tr>
1714
<tr><td>(t)op</td> </tr>
1715
</table></td> </tr>
1716
</table>"""
1717
1718
class HtmlDiff(object):
1719
"""For producing HTML side by side comparison with change highlights.
1720
1721
This class can be used to create an HTML table (or a complete HTML file
1722
containing the table) showing a side by side, line by line comparison
1723
of text with inter-line and intra-line change highlights. The table can
1724
be generated in either full or contextual difference mode.
1725
1726
The following methods are provided for HTML generation:
1727
1728
make_table -- generates HTML for a single side by side table
1729
make_file -- generates complete HTML file with a single side by side table
1730
1731
See Doc/includes/diff.py for an example usage of this class.
1732
"""
1733
1734
_file_template = _file_template
1735
_styles = _styles
1736
_table_template = _table_template
1737
_legend = _legend
1738
_default_prefix = 0
1739
1740
def __init__(self,tabsize=8,wrapcolumn=None,linejunk=None,
1741
charjunk=IS_CHARACTER_JUNK):
1742
"""HtmlDiff instance initializer
1743
1744
Arguments:
1745
tabsize -- tab stop spacing, defaults to 8.
1746
wrapcolumn -- column number where lines are broken and wrapped,
1747
defaults to None where lines are not wrapped.
1748
linejunk,charjunk -- keyword arguments passed into ndiff() (used by
1749
HtmlDiff() to generate the side by side HTML differences). See
1750
ndiff() documentation for argument default values and descriptions.
1751
"""
1752
self._tabsize = tabsize
1753
self._wrapcolumn = wrapcolumn
1754
self._linejunk = linejunk
1755
self._charjunk = charjunk
1756
1757
def make_file(self, fromlines, tolines, fromdesc='', todesc='',
1758
context=False, numlines=5, *, charset='utf-8'):
1759
"""Returns HTML file of side by side comparison with change highlights
1760
1761
Arguments:
1762
fromlines -- list of "from" lines
1763
tolines -- list of "to" lines
1764
fromdesc -- "from" file column header string
1765
todesc -- "to" file column header string
1766
context -- set to True for contextual differences (defaults to False
1767
which shows full differences).
1768
numlines -- number of context lines. When context is set True,
1769
controls number of lines displayed before and after the change.
1770
When context is False, controls the number of lines to place
1771
the "next" link anchors before the next change (so click of
1772
"next" link jumps to just before the change).
1773
charset -- charset of the HTML document
1774
"""
1775
1776
return (self._file_template % dict(
1777
styles=self._styles,
1778
legend=self._legend,
1779
table=self.make_table(fromlines, tolines, fromdesc, todesc,
1780
context=context, numlines=numlines),
1781
charset=charset
1782
)).encode(charset, 'xmlcharrefreplace').decode(charset)
1783
1784
def _tab_newline_replace(self,fromlines,tolines):
1785
"""Returns from/to line lists with tabs expanded and newlines removed.
1786
1787
Instead of tab characters being replaced by the number of spaces
1788
needed to fill in to the next tab stop, this function will fill
1789
the space with tab characters. This is done so that the difference
1790
algorithms can identify changes in a file when tabs are replaced by
1791
spaces and vice versa. At the end of the HTML generation, the tab
1792
characters will be replaced with a nonbreakable space.
1793
"""
1794
def expand_tabs(line):
1795
# hide real spaces
1796
line = line.replace(' ','\0')
1797
# expand tabs into spaces
1798
line = line.expandtabs(self._tabsize)
1799
# replace spaces from expanded tabs back into tab characters
1800
# (we'll replace them with markup after we do differencing)
1801
line = line.replace(' ','\t')
1802
return line.replace('\0',' ').rstrip('\n')
1803
fromlines = [expand_tabs(line) for line in fromlines]
1804
tolines = [expand_tabs(line) for line in tolines]
1805
return fromlines,tolines
1806
1807
def _split_line(self,data_list,line_num,text):
1808
"""Builds list of text lines by splitting text lines at wrap point
1809
1810
This function will determine if the input text line needs to be
1811
wrapped (split) into separate lines. If so, the first wrap point
1812
will be determined and the first line appended to the output
1813
text line list. This function is used recursively to handle
1814
the second part of the split line to further split it.
1815
"""
1816
# if blank line or context separator, just add it to the output list
1817
if not line_num:
1818
data_list.append((line_num,text))
1819
return
1820
1821
# if line text doesn't need wrapping, just add it to the output list
1822
size = len(text)
1823
max = self._wrapcolumn
1824
if (size <= max) or ((size -(text.count('\0')*3)) <= max):
1825
data_list.append((line_num,text))
1826
return
1827
1828
# scan text looking for the wrap point, keeping track if the wrap
1829
# point is inside markers
1830
i = 0
1831
n = 0
1832
mark = ''
1833
while n < max and i < size:
1834
if text[i] == '\0':
1835
i += 1
1836
mark = text[i]
1837
i += 1
1838
elif text[i] == '\1':
1839
i += 1
1840
mark = ''
1841
else:
1842
i += 1
1843
n += 1
1844
1845
# wrap point is inside text, break it up into separate lines
1846
line1 = text[:i]
1847
line2 = text[i:]
1848
1849
# if wrap point is inside markers, place end marker at end of first
1850
# line and start marker at beginning of second line because each
1851
# line will have its own table tag markup around it.
1852
if mark:
1853
line1 = line1 + '\1'
1854
line2 = '\0' + mark + line2
1855
1856
# tack on first line onto the output list
1857
data_list.append((line_num,line1))
1858
1859
# use this routine again to wrap the remaining text
1860
self._split_line(data_list,'>',line2)
1861
1862
def _line_wrapper(self,diffs):
1863
"""Returns iterator that splits (wraps) mdiff text lines"""
1864
1865
# pull from/to data and flags from mdiff iterator
1866
for fromdata,todata,flag in diffs:
1867
# check for context separators and pass them through
1868
if flag is None:
1869
yield fromdata,todata,flag
1870
continue
1871
(fromline,fromtext),(toline,totext) = fromdata,todata
1872
# for each from/to line split it at the wrap column to form
1873
# list of text lines.
1874
fromlist,tolist = [],[]
1875
self._split_line(fromlist,fromline,fromtext)
1876
self._split_line(tolist,toline,totext)
1877
# yield from/to line in pairs inserting blank lines as
1878
# necessary when one side has more wrapped lines
1879
while fromlist or tolist:
1880
if fromlist:
1881
fromdata = fromlist.pop(0)
1882
else:
1883
fromdata = ('',' ')
1884
if tolist:
1885
todata = tolist.pop(0)
1886
else:
1887
todata = ('',' ')
1888
yield fromdata,todata,flag
1889
1890
def _collect_lines(self,diffs):
1891
"""Collects mdiff output into separate lists
1892
1893
Before storing the mdiff from/to data into a list, it is converted
1894
into a single line of text with HTML markup.
1895
"""
1896
1897
fromlist,tolist,flaglist = [],[],[]
1898
# pull from/to data and flags from mdiff style iterator
1899
for fromdata,todata,flag in diffs:
1900
try:
1901
# store HTML markup of the lines into the lists
1902
fromlist.append(self._format_line(0,flag,*fromdata))
1903
tolist.append(self._format_line(1,flag,*todata))
1904
except TypeError:
1905
# exceptions occur for lines where context separators go
1906
fromlist.append(None)
1907
tolist.append(None)
1908
flaglist.append(flag)
1909
return fromlist,tolist,flaglist
1910
1911
def _format_line(self,side,flag,linenum,text):
1912
"""Returns HTML markup of "from" / "to" text lines
1913
1914
side -- 0 or 1 indicating "from" or "to" text
1915
flag -- indicates if difference on line
1916
linenum -- line number (used for line number column)
1917
text -- line text to be marked up
1918
"""
1919
try:
1920
linenum = '%d' % linenum
1921
id = ' id="%s%s"' % (self._prefix[side],linenum)
1922
except TypeError:
1923
# handle blank lines where linenum is '>' or ''
1924
id = ''
1925
# replace those things that would get confused with HTML symbols
1926
text=text.replace("&","&amp;").replace(">","&gt;").replace("<","&lt;")
1927
1928
# make space non-breakable so they don't get compressed or line wrapped
1929
text = text.replace(' ','&nbsp;').rstrip()
1930
1931
# add a class to the td tag if there is a difference on the line
1932
css_class = ' class="diff_changed" ' if flag else ' '
1933
1934
return f'<td class="diff_header"{id}>{linenum}</td>' \
1935
+ f'<td{css_class}nowrap="nowrap">{text}</td>'
1936
1937
def _make_prefix(self):
1938
"""Create unique anchor prefixes"""
1939
1940
# Generate a unique anchor prefix so multiple tables
1941
# can exist on the same HTML page without conflicts.
1942
fromprefix = "from%d_" % HtmlDiff._default_prefix
1943
toprefix = "to%d_" % HtmlDiff._default_prefix
1944
HtmlDiff._default_prefix += 1
1945
# store prefixes so line format method has access
1946
self._prefix = [fromprefix,toprefix]
1947
1948
def _convert_flags(self,fromlist,tolist,flaglist,context,numlines):
1949
"""Makes list of "next" links"""
1950
1951
# all anchor names will be generated using the unique "to" prefix
1952
toprefix = self._prefix[1]
1953
1954
# process change flags, generating middle column of next anchors/links
1955
next_id = ['']*len(flaglist)
1956
next_href = ['']*len(flaglist)
1957
num_chg, in_change = 0, False
1958
last = 0
1959
for i,flag in enumerate(flaglist):
1960
if flag:
1961
if not in_change:
1962
in_change = True
1963
last = i
1964
# at the beginning of a change, drop an anchor a few lines
1965
# (the context lines) before the change for the previous
1966
# link
1967
i = max([0,i-numlines])
1968
next_id[i] = ' id="difflib_chg_%s_%d"' % (toprefix,num_chg)
1969
# at the beginning of a change, drop a link to the next
1970
# change
1971
num_chg += 1
1972
next_href[last] = '<a href="#difflib_chg_%s_%d">n</a>' % (
1973
toprefix,num_chg)
1974
else:
1975
in_change = False
1976
# check for cases where there is no content to avoid exceptions
1977
if not flaglist:
1978
flaglist = [False]
1979
next_id = ['']
1980
next_href = ['']
1981
last = 0
1982
if context:
1983
fromlist = ['<td></td><td>&nbsp;No Differences Found&nbsp;</td>']
1984
tolist = fromlist
1985
else:
1986
fromlist = tolist = ['<td></td><td>&nbsp;Empty File&nbsp;</td>']
1987
# if not a change on first line, drop a link
1988
if not flaglist[0]:
1989
next_href[0] = '<a href="#difflib_chg_%s_0">f</a>' % toprefix
1990
# redo the last link to link to the top
1991
next_href[last] = '<a href="#difflib_chg_%s_top">t</a>' % (toprefix)
1992
1993
return fromlist,tolist,flaglist,next_href,next_id
1994
1995
def make_table(self,fromlines,tolines,fromdesc='',todesc='',context=False,
1996
numlines=5):
1997
"""Returns HTML table of side by side comparison with change highlights
1998
1999
Arguments:
2000
fromlines -- list of "from" lines
2001
tolines -- list of "to" lines
2002
fromdesc -- "from" file column header string
2003
todesc -- "to" file column header string
2004
context -- set to True for contextual differences (defaults to False
2005
which shows full differences).
2006
numlines -- number of context lines. When context is set True,
2007
controls number of lines displayed before and after the change.
2008
When context is False, controls the number of lines to place
2009
the "next" link anchors before the next change (so click of
2010
"next" link jumps to just before the change).
2011
"""
2012
2013
# make unique anchor prefixes so that multiple tables may exist
2014
# on the same page without conflict.
2015
self._make_prefix()
2016
2017
# change tabs to spaces before it gets more difficult after we insert
2018
# markup
2019
# it also removes trailing newlines, causing some diffs to be missed
2020
# see: gh-71896
2021
fromlines,tolines = self._tab_newline_replace(fromlines,tolines)
2022
2023
# create diffs iterator which generates side by side from/to data
2024
if context:
2025
context_lines = numlines
2026
else:
2027
context_lines = None
2028
diffs = _mdiff(fromlines,tolines,context_lines,linejunk=self._linejunk,
2029
charjunk=self._charjunk)
2030
2031
# set up iterator to wrap lines that exceed desired width
2032
if self._wrapcolumn:
2033
diffs = self._line_wrapper(diffs)
2034
2035
# collect up from/to lines and flags into lists (also format the lines)
2036
fromlist,tolist,flaglist = self._collect_lines(diffs)
2037
2038
# process change flags, generating middle column of next anchors/links
2039
fromlist,tolist,flaglist,next_href,next_id = self._convert_flags(
2040
fromlist,tolist,flaglist,context,numlines)
2041
2042
s = []
2043
fmt = ' <tr><td class="diff_next"%s>%s</td>%s' + \
2044
'<td class="diff_next">%s</td>%s</tr>\n'
2045
for i in range(len(flaglist)):
2046
if flaglist[i] is None:
2047
# mdiff yields None on separator lines skip the bogus ones
2048
# generated for the first line
2049
if i > 0:
2050
s.append(' </tbody> \n <tbody>\n')
2051
else:
2052
s.append( fmt % (next_id[i],next_href[i],fromlist[i],
2053
next_href[i],tolist[i]))
2054
if fromdesc or todesc:
2055
header_row = '<thead><tr>%s%s%s%s</tr></thead>' % (
2056
'<th class="diff_next"><br /></th>',
2057
'<th colspan="2" class="diff_header">%s</th>' % fromdesc,
2058
'<th class="diff_next"><br /></th>',
2059
'<th colspan="2" class="diff_header">%s</th>' % todesc)
2060
else:
2061
header_row = ''
2062
2063
table = self._table_template % dict(
2064
data_rows=''.join(s),
2065
header_row=header_row,
2066
prefix=self._prefix[1])
2067
2068
return table.replace('\0+','<span class="diff_add">'). \
2069
replace('\0-','<span class="diff_sub">'). \
2070
replace('\0^','<span class="diff_chg">'). \
2071
replace('\1','</span>'). \
2072
replace('\t','&nbsp;')
2073
2074
2075
def restore(delta, which):
2076
r"""
2077
Generate one of the two sequences that generated a delta.
2078
2079
Given a `delta` produced by `Differ.compare()` or `ndiff()`, extract
2080
lines originating from file 1 or 2 (parameter `which`), stripping off line
2081
prefixes.
2082
2083
Examples:
2084
2085
>>> diff = ndiff('one\ntwo\nthree\n'.splitlines(keepends=True),
2086
... 'ore\ntree\nemu\n'.splitlines(keepends=True))
2087
>>> diff = list(diff)
2088
>>> print(''.join(restore(diff, 1)), end="")
2089
one
2090
two
2091
three
2092
>>> print(''.join(restore(diff, 2)), end="")
2093
ore
2094
tree
2095
emu
2096
"""
2097
try:
2098
tag = {1: "- ", 2: "+ "}[int(which)]
2099
except KeyError:
2100
raise ValueError('unknown delta choice (must be 1 or 2): %r'
2101
% which) from None
2102
prefixes = (" ", tag)
2103
for line in delta:
2104
if line[:2] in prefixes:
2105
yield line[2:]