GHC 2016-12-17

25 comments.

, https://git.io/v15Iq in Homebrew/formula-patches
Goes with https://github.com/Homebrew/homebrew-core/pull/7973.

, https://git.io/v15Im in Homebrew/homebrew-core
python 2.7.13
=============

- [x] Have you followed the [guidelines for contributing](https://github.com/Homebrew/homebrew-core/blob/master/CONTRIBUTING.md)?
- [x] Have you checked that there aren't other open [pull requests](https://github.com/Homebrew/homebrew-core/pulls) for the same formula update/change?
- [x] Have you built your formula locally with `brew install --build-from-source <formula>`, where `<formula>` is the name of the formula you're submitting?
- [x] Does your build pass `brew audit --strict <formula>` (after doing `brew install <formula>`)?

-----

I'm updating setuptools, pip and wheel to the latest versions. Also pulling out the embedded tk patch into formula-patches (https://github.com/Homebrew/formula-patches/pull/97), because it makes the formula cleaner. Let me know if you want it back.

, https://git.io/v15It in Homebrew/formula-patches
python: patch for brewed tk
===========================

Existing patch pulled out of python.rb (previously embedded).

, https://git.io/v15IL in yyuu/pyenv
Add CPython 2.7.13
==================

https://www.python.org/downloads/release/python-2713/.

, https://git.io/v17hw in jarun/googler
Miscellaneous improvements
==========================

- Add -V, --version option (standard stuff);
- Reorder options in completion defs according to help text, so that checking whether any option's missing is easier;
- Add Debian and Ubuntu availability badges (can't give a version, so just point out which distro releases googler is available on, and link to respective package search pages).

We may want to cut a release some time soon.

, https://git.io/v17QW in jarun/googler
> if we set n and p we can't search a keyword starting using n and p.

Not sure what that means. You can certainly search keywords starting with n or p. The only prohibited keywords are the letters n and p and other commands, verbatim, which are not very useful keywords to begin with.

, https://git.io/v17QC in jarun/googler
You can figure out the raw input sequences arrow keys represent (if you ditch readline, otherwise they'll just be captured), but using arrow keys (or ^P, ^N, etc.) for control doesn't sound right outside curses.

, https://git.io/v17Qc in yyuu/pyenv
Add CPython 3.6.0rc2
====================

https://www.python.org/downloads/release/python-360rc2/.

, https://git.io/v17SQ in jarun/googler
> Also I' assuming that we didn't add this to omniprompt help because we show the message explicitly.

Yes, just like unfilter. The longer the help the less likely it is going to be read.

, https://git.io/v17S7 in jarun/googler
Oops...

, https://git.io/v172b in jarun/googler
Major changes are required to get this to work because the a tags don't have classes.

1. Need to start from span, and register a custom set of handlers;
2. Need to distinguish between the two cases and pass around more states;
3. More complexity at printing stage.

I don't feel like handling this case. More results doesn't hurt (and this should only happen when the the quality of the results for the original keywords isn't good), unlike results about something else.

, https://git.io/v172N in jarun/googler
Can't download as text. Put in a gist?

, https://git.io/v172F in jarun/googler
> "Including results for navy Search only for nevy"

When I search for nevy I only get results for nevy... Need the HTML from debug. I'll take a look tomorrow.

> Do you think it's reasonable to handle all these cases or leave it to the user to correct in the omniprompt... now that we have cmdline arg correction in place?

Not sure without digging in, but I dont' think including extra results hurt, and not showing auto-correction info in this case is at least justifiable.

, https://git.io/v17gR in jarun/googler
Pushed.

, https://git.io/v17gB in jarun/googler
Actually, we still have one color-neutral styling trick up our sleeves: the underline. Patch on top:

```diff
From 752af338ccfcc5af6d3822c658c7e65d94f4a6f9 Mon Sep 17 00:00:00 2001
From: Zhiming Wang <[email protected]>
Date: Sat, 17 Dec 2016 02:48:58 -0500
Subject: [PATCH] googler: underline autocorrected keyword(s)

---
 googler | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/googler b/googler
index 1768017..2fc3e68 100755
--- a/googler
+++ b/googler
@@ -1794,8 +1794,14 @@ class GooglerCmd(object):
         self.fetch()
         colors = self.colors
         if self._autocorrected_to:
+            if colors:
+                # Underline the keywords
+                print('hi')
+                autocorrected_to = '\x1b[4m' + self._autocorrected_to + '\x1b[24m'
+            else:
+                autocorrected_to = self._autocorrected_to
             autocorrect_info = ('Showing results for %s; enter "exact" for an exact search.' %
-                                self._autocorrected_to)
+                                autocorrected_to)
             printerr('')
             if colors:
                 printerr(colors.prompt + autocorrect_info + colors.reset)
-- 
2.11.0

```

I can apply this if you think it's better.

, https://git.io/v17g8 in jarun/googler
Just realized I pasted the wrong demo link...

https://asciinema.org/a/2veowecoo2hpznua9k1bgsnxj

The query above (demonstrating the quoting dilemma) is shown there.

, https://git.io/v17g4 in jarun/googler
> I checked the changes in regular as well as --noua mode. Both show same results with auto-correction warning. What's not working with --noua (you mentioned something)?

The code as I initially wrote it (following my understanding of the structure of that block, as documented in the new parser comments) didn't work under --noua, because the block's structure is different under --noua. See the added note under "User Agent disabled (differences)".

I have accounted for the difference, so the code works in whatever environment (hopefully).

> Also, how about highlighting the corrected spelling? At least within quotes?

I don't think we [have a suitable color for it](https://github.com/jarun/googler#colors) so coloring isn't an option. I thought about quotes but rejected that idea, because

```
$ googler -n 3 \"googler news\"

Showing results for "google news"; enter "exact" for an exact search.

1 Google News
https://news.google.com/
Comprehensive up-to-date news coverage, aggregated from sources all over the world by Google News.

2 Google News & Weather on the App Store - iTunes - Apple
https://itunes.apple.com/us/app/google-news-weather/id913753848?mt=8
Oct 3, 2016 - Your comprehensive and personalized view of headline stories and local news & weather. ° Coverage from
75,000 publications ◦ Instant-load ...

3 Google News - Mashable
http://mashable.com/category/google-news/
Google is killing Reader tomorrow, and as a sort of consolation prize, they've added some new features to Google
News. There's now a box that features a ...

googler (? for help) q
```

, https://git.io/v17gl in jarun/googler
> just trying to keep it low if we can.

I'm afraid we can't optimize here. There's a bit of nickle and diming you can do in the conditions at the cost of overall consistency in the code base. Not much you can do otherwise in a linear search for a bunch of things in rather hostile, poorly structured input.

To truly improve the sensitivity to one added condition, what you can do is scan the tree once, cache the results in some sort of search tree(s), then select with logarithmic (or better) complexity. It would be super hardcore though, and whether it will actually save time is questionable for what we do. I checked bs4 source code and it's simply doing a linear traversal for finding and selector queries; see [here](https://git.launchpad.net/~vivekchandola/beautifulsoup/+git/trunk/tree/library/bs4/element.py#n1296) and [here](https://git.launchpad.net/~vivekchandola/beautifulsoup/+git/trunk/tree/library/bs4/element.py#n1518). I don't know how JavaScript folks implement CSS selector queries (`querySelectorAll` and simple special cases).

Anyway, this consideration is not very helpful, since processing time is dwarfed by I/O.

, https://git.io/v17uu in jarun/googler
> Doesn't this condition alone indicate auto-correction?

It doesn't. See parser docs:

```
    #   1. When npfr=1 (exact), there could still be an
    #      a.spell, in a block that reads (English version) "Did you mean:
    #      ...". Therefore, we only consider the query autocorrected when a
    #      meaningful .spell_orig is also present (self.autocorrected).
```

> we do around 7000 pattern matches and 800+ tag parsing for a page on an average.

I recall a benchmark earlier demonstrating that I/O is the real bottleneck.

, https://git.io/v17uE in jarun/googler
👍 

, https://git.io/v17uR in jarun/googler
Addressed comments.

> we have ample scope to optimize our parser if we use elif

As mentioned in an inline comment, because we always return from if bodies, using elif hardly ever helps. Also, the conditions are seldom mutually exclusive (if ever), at a glance.

, https://git.io/v17u0 in jarun/googler
I'll throw a span tag here. In the beginning it was an a tag, just like `a.spell`, but that doesn't work with `--noua`.

, https://git.io/v17RX in jarun/googler
This "exact" command is like a poor man's version of the originally planned dynamic option manipulation (for the single option `--exact`). Just saying.

, https://git.io/v17R1 in jarun/googler
Fair enough, #155, similar to the how we handle filtered results. Heck, turned out to be a fair bit of work.

, https://git.io/v17RP in jarun/googler
Improve autocorrect handling: detect autocorrected query and add a prompt command "exact"
=========================================================================================

Details are in commit messages and additions to parser docs. This PR should be considered on a commit-by-commit basis, and each of the latter two commits (cdfe3d0 and ac6e13c) is optional, although I think they are worthy improvements.

Demo: https://asciinema.org/a/2veowecoo2hpznua9k1bgsnxj.

Closes #154.