[Reproducible-commits] [diffoscope] 20/21: Switch to Python 3
Jérémy Bobbio
lunar at moszumanska.debian.org
Mon Sep 21 17:39:29 UTC 2015
This is an automated email from the git hooks/post-receive script.
lunar pushed a commit to branch master
in repository diffoscope.
commit 84a58ee4000c2a55cf59eaef15bb894f8bbd5ede
Author: Jérémy Bobbio <lunar at debian.org>
Date: Mon Sep 21 16:18:51 2015 +0200
Switch to Python 3
This is the “red pill” commit where we jump from Python 2.7 to Python 3.3+:
* She-bang now calls /usr/bin/python3.
* debian/control file is updated to depends on python3-* packages.
* py.test-3 is called instead of pytest to run the test suit.
* We no longer need need to wrap sys.stdout as its output will be properly
encoded depending on the system settings.
* All “import __future__” statements are removed.
* items() replaces viewitems() and keys() replaces viewkeys() when working
on dicts.
https://docs.python.org/3/whatsnew/3.0.html#views-and-iterators-instead-of-lists
* We use the new metaclass syntax:
https://www.python.org/dev/peps/pep-3115/
* Msgunfmt.CHARSET_RE will work on raw bytes, so we need to add the 'b' modifier
to the regex string.
* str() replaces unicode() and chr() replaces unichr(): all strings are
unicode strings now.
https://docs.python.org/3/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit
* reduce() is no longer a built-in, so use functools.reduce() instead.
* TarContainer.extract doesn't need to decode the returned path as all paths
are now unicode strings.
* We have to tweak the expected output when testing the RPM header conversion
because repr() no longer include a trailing L anymore.
https://docs.python.org/3/whatsnew/3.0.html#integers
---
README | 2 +-
bin/diffoscope | 2 +-
debian/control | 22 +++++++++++-----------
debian/rules | 2 +-
debian/tests/control | 2 +-
debian/tests/pytest | 2 +-
diffoscope/__main__.py | 6 +-----
diffoscope/changes.py | 2 --
diffoscope/comparators/__init__.py | 4 ++--
diffoscope/comparators/binary.py | 4 +---
diffoscope/comparators/deb.py | 2 --
diffoscope/comparators/gettext.py | 2 +-
diffoscope/comparators/haskell.py | 2 +-
diffoscope/comparators/libarchive.py | 1 -
diffoscope/comparators/rpm.py | 8 +++++---
diffoscope/comparators/tar.py | 2 +-
diffoscope/comparators/utils.py | 18 ++++++------------
diffoscope/presenters/html.py | 6 +++---
setup.py | 4 ++--
tests/comparators/test_debian.py | 2 --
tests/comparators/test_directory.py | 2 --
tests/data/rpm_header_expected_diff | 34 +++++++++++++++++-----------------
22 files changed, 56 insertions(+), 75 deletions(-)
diff --git a/README b/README
index f7e3e7c..3587c6c 100644
--- a/README
+++ b/README
@@ -32,7 +32,7 @@ This will compare `build1.changes` and `build2.changes` and create
External dependencies
---------------------
-Required Python modules:
+diffoscope requires Python 3 and the following modules:
* Available on PyPI: libarchive-c, python-debian.
* Magic-file-extension is built from file:
diff --git a/bin/diffoscope b/bin/diffoscope
index 1ac89a8..2393807 100755
--- a/bin/diffoscope
+++ b/bin/diffoscope
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# diffoscope: in-depth comparison of files, archives, and directories
diff --git a/debian/control b/debian/control
index 2673149..eb26989 100644
--- a/debian/control
+++ b/debian/control
@@ -7,18 +7,18 @@ Uploaders: Jérémy Bobbio <lunar at debian.org>,
Holger Levsen <holger at debian.org>
Build-Depends: debhelper (>= 9),
dh-python,
- python-all,
- python-debian,
- python-docutils,
- python-libarchive-c,
- python-magic,
- python-pytest,
- python-rpm,
- python-setuptools,
- python-tlsh,
+ python3-all,
+ python3-debian,
+ python3-docutils,
+ python3-libarchive-c,
+ python3-magic,
+ python3-pytest,
+ python3-rpm,
+ python3-setuptools,
+ python3-tlsh,
rpm-common
Standards-Version: 3.9.6
-X-Python-Version: >= 2.7
+X-Python-Version: >= 3
Homepage: http://diffoscope.org/
Vcs-Git: git://anonscm.debian.org/reproducible/diffoscope.git
Vcs-Browser: https://anonscm.debian.org/cgit/reproducible/diffoscope.git
@@ -27,7 +27,7 @@ XS-Testsuite: autopkgtest
Package: diffoscope
Architecture: all
Depends: ${misc:Depends},
- ${python:Depends},
+ ${python3:Depends},
Recommends: ${diffoscope:Recommends}
Breaks: debbindiff (<< 29)
Replaces: debbindiff (<< 29)
diff --git a/debian/rules b/debian/rules
index 4595b44..8a5f15c 100755
--- a/debian/rules
+++ b/debian/rules
@@ -3,7 +3,7 @@
VERSION = $(shell dpkg-parsechangelog --show-field Version)
%:
- dh $@ --with python2 --buildsystem=pybuild
+ dh $@ --with python3 --buildsystem=pybuild
override_dh_auto_build:
@if ! grep -q 'VERSION = "$(VERSION)"' diffoscope/__init__.py; then \
diff --git a/debian/tests/control b/debian/tests/control
index ead0816..7819245 100644
--- a/debian/tests/control
+++ b/debian/tests/control
@@ -1,3 +1,3 @@
Tests: pytest
-Depends: diffoscope, python-pytest
+Depends: diffoscope, python3-pytest
Restrictions: needs-recommends
diff --git a/debian/tests/pytest b/debian/tests/pytest
index 0c398a5..38b6eee 100755
--- a/debian/tests/pytest
+++ b/debian/tests/pytest
@@ -8,4 +8,4 @@ if ! [ -d "$ADTTMP" ]; then
fi
cp -r tests $ADTTMP
-(cd $ADTTMP; py.test)
+(cd $ADTTMP; py.test-3)
diff --git a/diffoscope/__main__.py b/diffoscope/__main__.py
index 6971c2f..d0ed2df 100644
--- a/diffoscope/__main__.py
+++ b/diffoscope/__main__.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# diffoscope: in-depth comparison of files, archives, and directories
@@ -18,8 +18,6 @@
# You should have received a copy of the GNU General Public License
# along with diffoscope. If not, see <http://www.gnu.org/licenses/>.
-from __future__ import print_function
-
import argparse
from contextlib import contextmanager
import logging
@@ -82,8 +80,6 @@ def create_parser():
def make_printer(path):
if path == '-':
output = sys.stdout
- if not sys.stdout.isatty():
- output = codecs.getwriter(sys.stdin.encoding or 'utf-8')(sys.stdout)
else:
output = codecs.open(path, 'w', encoding='utf-8')
def print_func(*args, **kwargs):
diff --git a/diffoscope/changes.py b/diffoscope/changes.py
index 43c4836..1459176 100644
--- a/diffoscope/changes.py
+++ b/diffoscope/changes.py
@@ -37,8 +37,6 @@ code is copyright (c) Jonny Lamb, and is used by dput, rather then created as
a result of it. Thank you Jonny.
"""
-from __future__ import print_function
-
__author__ = 'Jonny Lamb'
__copyright__ = 'Copyright © 2008 Jonny Lamb, Copyright © 2010 Jan Dittberner'
__license__ = 'MIT'
diff --git a/diffoscope/comparators/__init__.py b/diffoscope/comparators/__init__.py
index 3a6d0d6..9f9f3a1 100644
--- a/diffoscope/comparators/__init__.py
+++ b/diffoscope/comparators/__init__.py
@@ -161,11 +161,11 @@ def perform_fuzzy_matching(members1, members2):
# Perform local copies because they will be modified by consumer
members1 = dict(members1)
members2 = dict(members2)
- for name1, file1 in members1.viewitems():
+ for name1, file1 in members1.items():
if file1.is_directory() or not file1.fuzzy_hash:
continue
comparisons = []
- for name2, file2 in members2.viewitems():
+ for name2, file2 in members2.items():
if name2 in already_compared or file2.is_directory() or not file2.fuzzy_hash:
continue
comparisons.append((tlsh.diff(file1.fuzzy_hash, file2.fuzzy_hash), name2))
diff --git a/diffoscope/comparators/binary.py b/diffoscope/comparators/binary.py
index fea9975..b156ec6 100644
--- a/diffoscope/comparators/binary.py
+++ b/diffoscope/comparators/binary.py
@@ -78,9 +78,7 @@ def needs_content(original_method):
return original_method(self, other, *args, **kwargs)
return wrapper
-class File(object):
- __metaclass__ = ABCMeta
-
+class File(object, metaclass=ABCMeta):
@classmethod
def guess_file_type(self, path):
if not hasattr(self, '_mimedb'):
diff --git a/diffoscope/comparators/deb.py b/diffoscope/comparators/deb.py
index 3dd4dbd..5b8e36e 100644
--- a/diffoscope/comparators/deb.py
+++ b/diffoscope/comparators/deb.py
@@ -17,8 +17,6 @@
# You should have received a copy of the GNU General Public License
# along with diffoscope. If not, see <http://www.gnu.org/licenses/>.
-from __future__ import absolute_import
-
import re
import os.path
from debian.arfile import ArFile
diff --git a/diffoscope/comparators/gettext.py b/diffoscope/comparators/gettext.py
index 4e02dea..d5812d7 100644
--- a/diffoscope/comparators/gettext.py
+++ b/diffoscope/comparators/gettext.py
@@ -27,7 +27,7 @@ from diffoscope import logger
class Msgunfmt(Command):
- CHARSET_RE = re.compile(r'^"Content-Type: [^;]+; charset=([^\\]+)\\n"$')
+ CHARSET_RE = re.compile(rb'^"Content-Type: [^;]+; charset=([^\\]+)\\n"$')
def __init__(self, *args, **kwargs):
super(Msgunfmt, self).__init__(*args, **kwargs)
diff --git a/diffoscope/comparators/haskell.py b/diffoscope/comparators/haskell.py
index 5f12914..b4c0a9b 100644
--- a/diffoscope/comparators/haskell.py
+++ b/diffoscope/comparators/haskell.py
@@ -76,7 +76,7 @@ class HiFile(File):
fp.read(4)
# now read version
buf = fp.read(16)
- version_found = ''.join(map(unichr, struct.unpack_from('=3IB', buf)))
+ version_found = ''.join(map(chr, struct.unpack_from('=3IB', buf)))
if version_found != HiFile.hi_version:
logger.debug('Haskell version mismatch. Found %s instead of %s.',
version_found, HiFile.hi_version)
diff --git a/diffoscope/comparators/libarchive.py b/diffoscope/comparators/libarchive.py
index e0b8960..8d8537b 100644
--- a/diffoscope/comparators/libarchive.py
+++ b/diffoscope/comparators/libarchive.py
@@ -17,7 +17,6 @@
# You should have received a copy of the GNU General Public License
# along with diffoscope. If not, see <http://www.gnu.org/licenses/>.
-from __future__ import absolute_import
from contextlib import contextmanager
import ctypes
diff --git a/diffoscope/comparators/rpm.py b/diffoscope/comparators/rpm.py
index b8aa299..641dac1 100644
--- a/diffoscope/comparators/rpm.py
+++ b/diffoscope/comparators/rpm.py
@@ -18,7 +18,7 @@
# You should have received a copy of the GNU General Public License
# along with diffoscope. If not, see <http://www.gnu.org/licenses/>.
-from __future__ import absolute_import
+from binascii import hexlify
from io import StringIO
import os.path
import subprocess
@@ -39,12 +39,14 @@ def convert_header_field(io, header):
io.write(u" - ")
convert_header_field(io, item)
elif isinstance(header, str):
+ io.write(header)
+ elif isinstance(header, bytes):
try:
io.write(header.decode('utf-8'))
except UnicodeDecodeError:
- io.write(header.encode('hex_codec').decode('us-ascii'))
+ io.write(hexlify(header).decode('us-ascii'))
else:
- io.write(repr(header).decode('us-ascii'))
+ io.write(repr(header))
def get_rpm_header(path, ts):
diff --git a/diffoscope/comparators/tar.py b/diffoscope/comparators/tar.py
index b45a1cd..27d4782 100644
--- a/diffoscope/comparators/tar.py
+++ b/diffoscope/comparators/tar.py
@@ -105,7 +105,7 @@ class TarContainer(Archive):
def extract(self, member_name, dest_dir):
logger.debug('tar extracting %s to %s', member_name, dest_dir)
self.archive.extract(member_name, dest_dir)
- return os.path.join(dest_dir, member_name).decode('utf-8')
+ return os.path.join(dest_dir, member_name)
def get_member(self, member_name):
tarinfo = self.archive.getmember(member_name)
diff --git a/diffoscope/comparators/utils.py b/diffoscope/comparators/utils.py
index 6364abc..2cff408 100644
--- a/diffoscope/comparators/utils.py
+++ b/diffoscope/comparators/utils.py
@@ -52,9 +52,7 @@ def get_ar_content(path):
['ar', 'tv', path], stderr=subprocess.STDOUT, shell=False).decode('utf-8')
-class Command(object):
- __metaclass__ = ABCMeta
-
+class Command(object, metaclass=ABCMeta):
def __init__(self, path):
self._path = path
logger.debug('running %s', self.cmdline())
@@ -157,9 +155,7 @@ def get_compressed_content_name(path, expected_extension):
NO_COMMENT = None
-class Container(object):
- __metaclass__ = ABCMeta
-
+class Container(object, metaclass=ABCMeta):
def __init__(self, source):
self._source = source
@@ -186,16 +182,16 @@ class Container(object):
def comparisons(self, other):
my_members = self.get_members()
other_members = other.get_members()
- for name in sorted(my_members.viewkeys() & other_members.viewkeys()):
+ for name in sorted(my_members.keys() & other_members.keys()):
yield my_members.pop(name), other_members.pop(name), NO_COMMENT
for my_name, other_name, score in diffoscope.comparators.perform_fuzzy_matching(my_members, other_members):
comment = 'Files similar despite different names (difference score: %d)' % score
yield my_members.pop(my_name), other_members.pop(other_name), comment
if Config.general.new_file:
- for my_name in my_members.viewkeys() - other_members.viewkeys():
+ for my_name in my_members.keys() - other_members.keys():
my_file = my_members[my_name]
yield my_file, NonExistingFile('/dev/null', my_file), NO_COMMENT
- for other_name in other_members.viewkeys() - my_members.viewkeys():
+ for other_name in other_members.keys() - my_members.keys():
other_file = other_members[other_name]
yield NonExistingFile('/dev/null', other_file), other_file, NO_COMMENT
@@ -239,9 +235,7 @@ class ArchiveMember(File):
return False
-class Archive(Container):
- __metaclass__ = ABCMeta
-
+class Archive(Container, metaclass=ABCMeta):
def __init__(self, *args, **kwargs):
super(Archive, self).__init__(*args, **kwargs)
self._archive = None
diff --git a/diffoscope/presenters/html.py b/diffoscope/presenters/html.py
index 28c5446..42f7acd 100644
--- a/diffoscope/presenters/html.py
+++ b/diffoscope/presenters/html.py
@@ -31,7 +31,6 @@
# Dave Burt <dave (at) burt.id.au> (mainly for html theme)
#
-from __future__ import print_function
import cgi
import re
import sys
@@ -39,6 +38,7 @@ from xml.sax.saxutils import escape
from diffoscope import logger, VERSION
from diffoscope.config import Config
from diffoscope.presenters.icon import FAVICON_BASE64
+from functools import reduce
# minimum line size, we add a zero-sized breakable space every
# LINESIZE characters
@@ -188,9 +188,9 @@ def linediff(s, t):
Original line diff algorithm of diff2html. It's character based.
'''
if len(s):
- s = unicode(reduce(lambda x, y:x+y, [ sane(c) for c in s ]))
+ s = str(reduce(lambda x, y:x+y, [ sane(c) for c in s ]))
if len(t):
- t = unicode(reduce(lambda x, y:x+y, [ sane(c) for c in t ]))
+ t = str(reduce(lambda x, y:x+y, [ sane(c) for c in t ]))
m, n = len(s), len(t)
d = [[(0, 0) for i in range(n+1)] for i in range(m+1)]
diff --git a/setup.py b/setup.py
index f21fcc0..62cc5f9 100644
--- a/setup.py
+++ b/setup.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
import sys
from setuptools import setup, find_packages
@@ -26,7 +26,7 @@ class PyTest(TestCommand):
setup(name='diffoscope',
version=diffoscope.VERSION,
description='display differences between files',
- long_description=open('README').read(),
+ long_description=open('README', encoding='utf-8').read(),
author='Lunar',
author_email='lunar at debian.org',
url='https://wiki.debian.org/ReproducibleBuilds',
diff --git a/tests/comparators/test_debian.py b/tests/comparators/test_debian.py
index 60aba44..4e33e6b 100644
--- a/tests/comparators/test_debian.py
+++ b/tests/comparators/test_debian.py
@@ -17,8 +17,6 @@
# You should have received a copy of the GNU General Public License
# along with diffoscope. If not, see <http://www.gnu.org/licenses/>.
-from __future__ import print_function
-
import os.path
import shutil
import pytest
diff --git a/tests/comparators/test_directory.py b/tests/comparators/test_directory.py
index 269b2b7..d1008f7 100644
--- a/tests/comparators/test_directory.py
+++ b/tests/comparators/test_directory.py
@@ -17,8 +17,6 @@
# You should have received a copy of the GNU General Public License
# along with diffoscope. If not, see <http://www.gnu.org/licenses/>.
-from __future__ import print_function
-
import os
import os.path
import shutil
diff --git a/tests/data/rpm_header_expected_diff b/tests/data/rpm_header_expected_diff
index 27cfef6..d8729d0 100644
--- a/tests/data/rpm_header_expected_diff
+++ b/tests/data/rpm_header_expected_diff
@@ -3,10 +3,10 @@
+HEADERIMMUTABLE: 00000030000001b00000003f00000007000001a00000001000000064000000080000000000000001000003e8000000060000000200000001000003e9000000060000000700000001000003ea000000060000000900000001000003ec000000090000000b00000001000003ed000000090000002700000001000003ee000000040000003400000001000003ef000000060000003800000001000003f1000000040000004000000001000003f6000000060000004400000001000003f8000000090000004b00000001000003fd000000060000005f00000001000003fe0000000600000065000000010000040400 [...]
HEADERI18NTABLE:
- C
--SIGSIZE: 1583L
+-SIGSIZE: 1583
-SIGMD5: 06166011d2d66f8433d50b417358a31d
-SHA1HEADER: 331667229aa7e4d600f682e9cd0c6372b93b082a
-+SIGSIZE: 1722L
++SIGSIZE: 1722
+SIGMD5: 746686e44fc68a5836e8043f722e436a
+SHA1HEADER: 2780015bd7bd365949d05e29618fa833e62e4604
NAME: test
@@ -14,45 +14,45 @@
RELEASE: 0
SUMMARY: Test package for debbindiff
DESCRIPTION: Test package
--BUILDTIME: 1435168523L
-+BUILDTIME: 1435168561L
+-BUILDTIME: 1435168523
++BUILDTIME: 1435168561
BUILDHOST: loar
--SIZE: 446L
-+SIZE: 671L
+-SIZE: 446
++SIZE: 671
LICENSE: Public
GROUP: Applications/System
OS: linux
ARCH: x86_64
FILESIZES:
-- - 446L
-+ - 671L
+- - 446
++ - 671
FILEMODES:
- - 33188L
+ - 33188
FILERDEVS:
- - 0L
+ - 0
FILEMTIMES:
-- - 1435168518L
-+ - 1435168559L
+- - 1435168518
++ - 1435168559
FILEMD5S:
- - 7873b606429a70284d63f0674e6787bc
+ - 17866262a0c60d916e3cf3219724b2e7
FILELINKTOS:
-
FILEFLAGS:
- - 0L
+ - 0
FILEUSERNAME:
- root
FILEGROUPNAME:
- root
SOURCERPM: test-0-0.src.rpm
FILEVERIFYFLAGS:
- - 4294967295L
--ARCHIVESIZE: 696L
-+ARCHIVESIZE: 920L
+ - 4294967295
+-ARCHIVESIZE: 696
++ARCHIVESIZE: 920
PROVIDES:
- test - test(x86-64)
REQUIREFLAGS:
- - 16777226L - 16777226L
+ - 16777226 - 16777226
REQUIRES:
- rpmlib(CompressedFileNames) - rpmlib(PayloadFilesHavePrefix)
REQUIREVERSION:
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/reproducible/diffoscope.git
More information about the Reproducible-commits
mailing list