[Reportbug-maint] Bug#531911: More comments and possibly better fix

[chaica]

Hi Alex,

Could you send your patch again ? For some reasons the end of several
lines were truncated. Maybe you could send it again as an attachment.

Bye,
Carl Chenet

> First, sorry for the late answer, but the Debian BTS does not send
> mail to the bug submitter unless they subscribed to the bug report,
> and I forgot on this one, that is why I saw your follow-ups only
> today...
> 
> > I think the line is truncated to print the information given your shell
> > size. Can you confirm it works if you use a shell allowing the whole
> > line beeing printed (or if you avoid truncating at this exact spot) ?
> > For me it works with this workaround. Meanwhile I'm looking for how to
> > avoid a line being truncated this way.
> 
> Yeah, it works with this workaround.
> 
> > It seem this bug is related to the Python decode() function which seems
> > to behave weirdly given the default shell size (with other shell sizes,
> > the display is ok, no exception is raised). Using "replace" mode for
> > this function allows the character beeing replaced by the "?" character
> > in order to go on with the bug list display. That's the way reportbug
> > already deals with the sister function encode().
> 
> I do not think decode() takes into account shell size because it
> handles a string, not terminal output. replace() works around the bug
> but now, this is the output I get :
>   57) #522628  gnome-terminal: duplicate access keys for menu bar: “Terminal�
> 
> I think the attached patch is better, because it gives me the following output :
>   57) #522628  gnome-terminal: duplicate access keys for menu bar: “Terminal”,
> 
> Thanks,
> 
> Alex
> 
> The patch :
> 
> diff --git a/reportbug/debianbts.py b/reportbug/debianbts.py
> index 169e4c9..212a7ee 100644
> --- a/reportbug/debianbts.py
> +++ b/reportbug/debianbts.py
> @@ -929,6 +929,11 @@ def get_cgi_reports(package, system='debian', http_proxy=''
> 
>      parser = BTSParser(cgi=True)
>      for line in page:
> +        try:
> +            line = line.decode('utf-8') # BTS pages are encoded in utf-8
> +        except UnicodeDecodeError:
> +            # page has a bad char
> +            line = line.decode('utf-8', 'replace')
>          parser.feed(line)
>      parser.close()
>      try:
> diff --git a/reportbug/ui/text_ui.py b/reportbug/ui/text_ui.py
> index 15db2b7..e6f7360 100644
> --- a/reportbug/ui/text_ui.py
> +++ b/reportbug/ui/text_ui.py
> @@ -613,7 +613,7 @@ def browse_bugs(hierarchy, count, bugs, bts, queryonly, mirr
> 
>                  while 1:
>                      for line in lastpage:
> -                        sys.stderr.write(line.decode('utf-8', "replace").encode
> +                        sys.stderr.write(line.encode(output_encoding, "replace"
>                      x = select_options(pstr, options, helptext,
>                                         allow_numbers=allowed)
>                      if x == 'n':
> 
>