[18.10] /usr/sbin/cntlm incompatible?

Hello,

As I am trying to integrate a UM 18.10 VM onto the corporate network I am required to install/use CNTLM so I can access the corporate proxy and then use my VM.

Before going further, I would like to mention that I have probably already checked most of the online references (askubuntu, serverfault, superuser, linuxforums, etc) for this particular problem and even though I was able to learn more I am still stuck and hence now think it could be UM 18.10 related/isolated.

Here are the macro steps that I have covered in order to get CNTLM going:

  1. Installed cntlm and libc6 (it's req);
    Ref.: http://cntlm.sourceforge.net/

  2. Generated my hashes using cntlm -H;

  3. Updated /etc/cntlm.conf with new informations;
    Ref.: https://stackoverflow.com/questions/9181637/how-to-fill-in-proxy-information-in-cntlm-config-file

  4. Restarted/tested.

Here is how it looked like from a terminal perspective:

The package is properly installed:

$ which cntlm 
/usr/sbin/cntlm

Let's understand CNTLM's options (FYI):

$ /usr/sbin/cntlm -h
CNTLM - Accelerating NTLM Authentication Proxy version 0.92.3
Copyright (c) 2oo7-2o1o David Kubicek

This program comes with NO WARRANTY, to the extent permitted by law. You
may redistribute copies of it under the terms of the GNU GPL Version 2 or
newer. For more information about these matters, see the file LICENSE.
For copyright holders of included encryption routines see headers.

Usage: /usr/sbin/cntlm [-AaBcDdFfgHhILlMPpSsTUuvw] [:] ...
	-A  [/]
	    ACL allow rule. IP or hostname, net must be a number (CIDR notation)
	-a  ntlm | nt | lm
	    Authentication type - combined NTLM, just LM, or just NT. Default NTLM.
	    It is the most versatile setting and likely to work for you.
	-B  Enable NTLM-to-basic authentication.
	-c  
	    Configuration file. Other arguments can be used as well, overriding
	    config file settings.
	-D  [/]
	    ACL deny rule. Syntax same as -A.
	-d  
	    Domain/workgroup can be set separately.
	-f  Run in foreground, do not fork into daemon mode.
	-F  
	    NTLM authentication flags.
	-G  
	    User-Agent matching for the trans-isa-scan plugin.
	-g  Gateway mode - listen on all interfaces, not only loopback.
	-H  Print password hashes for use in config file (NTLMv2 needs -u and -d).
	-h  Print this help info along with version number.
	-I  Prompt for the password interactively.
	-L  [:]::
	    Forwarding/tunneling a la OpenSSH. Same syntax - listen on lport
	    and forward all connections through the proxy to rhost:rport.
	    Can be used for direct tunneling without corkscrew, etc.
	-l  [:]
	    Main listening port for the NTLM proxy.
	-M  
	    Magic autodetection of proxy's NTLM dialect.
	-N  "[, "
	    List of URL's to serve direcly as stand-alone proxy (e.g. '*.local')
	-O  [:]
	    Enable SOCKS5 proxy on port lport (binding to address saddr)
	-P  
	    Create a PID file upon successful start.
	-p  
	    Account password. Will not be visible in "ps", /proc, etc.
	-r  "HeaderName: value"
	    Add a header substitution. All such headers will be added/replaced
	    in the client's requests.
	-S  
	    Enable automation of GFI WebMonitor ISA scanner for files < size_in_kb.
	-s  Do not use threads, serialize all requests - for debugging only.
	-U  
	    Run as uid. It is an important security measure not to run as root.
	-u  [@<domain]
	    Domain/workgroup can be set separately.
	-v  Print debugging information.
	-w  
	    Some proxies require correct NetBIOS hostname.
$

Before deploying, let's create NTLM compatibles hashes and then apply them in /etc/cntlm.conf:

$ cntlm -u USERNAME -d CORPDOMAIN -H
Password: 
PassLM          97B1A354A90C182DAC8257D627CAF75F
PassNT          FA50CC02F904F07A6974E23EAE20E660
PassNTLMv2      25123DF21534B37C15FD77AA87B2CE97    # Only for user 'USERNAME', domain 'CORPDOMAIN'
$ sudo vi /etc/cntlm.conf
#
# Cntlm Authentication Proxy Configuration
#
# NOTE: all values are parsed literally, do NOT escape spaces,
# do not quote. Use 0600 perms if you use plaintext password.
#

Username       USERNAME
Domain          CORPDOMAIN

# NOTE: Use plaintext password only at your own risk
# Use hashes instead. You can use a "cntlm -M" and "cntlm -H"
# command sequence to get the right config for your environment.
# See cntlm man page
# Example secure config shown below.
# PassLM          1AD35398BE6565DDB5C4EF70C0593492
# PassNT          77B9081511704EE852F94227CF48A793

# Only for user 'USERNAME', domain 'CORPDOMAIN'
PassLM          97B1A354A90C182DAC8257D627CAF75F
PassNT          FA50CC02F904F07A6974E23EAE20E660
PassNTLMv2      25123DF21534B37C15FD77AA87B2CE97

# Specify the netbios hostname cntlm will send to the parent
# proxies. Normally the value is auto-guessed.
#
# Workstation   netbios_hostname

# List of parent proxies to use. More proxies can be defined
# one per line in format :
#
Proxy                    PROXY1HOST:PROXY1PORT
Proxy                    PROXY2HOST:PROXY2PORT

# List addresses you do not want to pass to parent proxies
# * and ? wildcards can be used
#
NoProxy         localhost, 127.0.0.*, 10.*, 192.168.*, 192.197.*, 172.16.*

# Specify the port cntlm will listen on
# You can bind cntlm to specific interface by specifying
# the appropriate IP address also in format :
# Cntlm listens on 127.0.0.1:3128 by default
#
Listen          9669

# If you wish to use the SOCKS5 proxy feature as well, uncomment
# the following option. It can be used several times
# to have SOCKS5 on more than one port or on different network
# interfaces (specify explicit source address for that).
#
# WARNING: The service accepts all requests, unless you use
# SOCKS5User and make authentication mandatory. SOCKS5User
# can be used repeatedly for a whole bunch of individual accounts.
#
#SOCKS5Proxy    8010
#SOCKS5User     dave:password

# Use -M first to detect the best NTLM settings for your proxy.
# Default is to use the only secure hash, NTLMv2, but it is not
# as available as the older stuff.
#
# This example is the most universal setup known to man, but it
# uses the weakest hash ever. I won't have it's usage on my
# conscience. :) Really, try -M first.
#
#Auth           LM
#Flags          0x06820000

# Enable to allow access from other computers
#
#Gateway        yes

# Useful in Gateway mode to allow/restrict certain IPs
# Specifiy individual IPs or subnets one rule per line.
#
#Allow          127.0.0.1
#Deny           0/0

# GFI WebMonitor-handling plugin parameters, disabled by default
#
#ISAScannerSize     1024
#ISAScannerAgent    Wget/
#ISAScannerAgent    APT-HTTP/
#ISAScannerAgent    Yum/

# Headers which should be replaced if present in the request
#
#Header         User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)

# Tunnels mapping local port to a machine behind the proxy.
# The format is ::
#
#Tunnel         11443:remote.com:443
:wq
$

As we can see from above, the only options that were added were:
Username USERNAME
Domain CORPDOMAIN

Only for user 'USERNAME', domain 'CORPDOMAIN'

PassLM 97B1A354A90C182DAC8257D627CAF75F
PassNT FA50CC02F904F07A6974E23EAE20E660
PassNTLMv2 25123DF21534B37C15FD77AA87B2CE97
Proxy PROXY1HOST:PROXY1PORT
Proxy PROXY2HOST:PROXY2PORT
Listen 9669
I have also appended some LAN prefixes @ "NoProxy localhost, 127.0.0., 10., 192.168., 192.197., 172.16.*" but that is irrelevant.

Let's start the service and see if it connects:

$ sudo cntlm -u USERNAME -I -v
[sudo] password for USERNAME: 
section: global, Domain = 'CORPDOMAIN'
section: global, PassLM = '97B1A354A90C182DAC8257D627CAF75F'
section: global, PassNT = 'FA50CC02F904F07A6974E23EAE20E660'
section: global, PassNTLMv2 = '25123DF21534B37C15FD77AA87B2CE97'
section: global, Proxy = 'PROXY1HOST:PROXY1PORT'
section: global, Proxy = 'PROXY2HOST:PROXY2PORT'
section: global, NoProxy = 'localhost, 127.0.0.*, 10.*, 192.168.*, 192.197.*, 172.16.*'
section: global, Listen = '9669'
Default config file opened successfully
cntlm: Cannot bind port 9669: Address already in use!
Adding no-proxy for: 'localhost'
Adding no-proxy for: '127.0.0.*'
Adding no-proxy for: '10.*'
Adding no-proxy for: '192.168.*'
Adding no-proxy for: '192.197.*'
Adding no-proxy for: '172.16.*'
No proxy service ports were successfully opened.
Exitting with error. Check daemon logs or run with -v.
$

From what we can see above it loaded but there is an error? Also, we have already specified -v so this echo doesn't make sense.

And According to UM 18.10 CNTLM is running:

$ sudo service cntlm status
[sudo] password for USERNAME: 
● cntlm.service - LSB: Authenticating HTTP accelerator for NTLM secured proxies
   Loaded: loaded (/etc/init.d/cntlm; generated)
   Active: active (running) since Mon 2019-03-11 13:33:51 EDT; 44min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 753 ExecStart=/etc/init.d/cntlm start (code=exited, status=0/SUCCESS)
    Tasks: 1 (limit: 4659)
   Memory: 1.1M
   CGroup: /system.slice/cntlm.service
           └─785 /usr/sbin/cntlm -U cntlm -P /var/run/cntlm/cntlm.pid
[...]
$ cat /var/run/cntlm/cntlm.pid 
785
$ ps -ef | grep -i 785
cntlm      785     1  0 13:33 ?        00:00:00 /usr/sbin/cntlm -U cntlm -P /var/run/cntlm/cntlm.pid
USERNAME   3406  3298  0 14:06 pts/2    00:00:00 grep --color=auto -i 785
$ sudo ss -tulpn | grep :9669
tcp    LISTEN   0        5               127.0.0.1:9669           0.0.0.0:*      users:(("cntlm",pid=785,fd=4))   
$

But apt is still unable to authenticate on the proxy:

$ sudo apt update
Err:1 http://archive.canonical.com/ubuntu cosmic InRelease
  407  Proxy Authentication Required [IP: PROXYHOST PROXYPORT]
Err:2 http://security.ubuntu.com/ubuntu cosmic-security InRelease
  407  Proxy Authentication Required [IP: PROXYHOST PROXYPORT]
Err:3 http://ca.archive.ubuntu.com/ubuntu cosmic InRelease
  407  Proxy Authentication Required [IP: PROXYHOST PROXYPORT]
Err:4 http://ca.archive.ubuntu.com/ubuntu cosmic-updates InRelease
  407  Proxy Authentication Required [IP: PROXYHOST PROXYPORT]
Err:5 http://ca.archive.ubuntu.com/ubuntu cosmic-backports InRelease
  407  Proxy Authentication Required [IP: PROXYHOST PROXYPORT]
Reading package lists... Done

So my question is: did anybody ever used CNTLM recently? If so, would you happen to have any tips and hints as right now I am blaming UM and (sadly) am considering deploying another flavor.

Please advise/Thanks in advance,

DLS

PS:

$ uname -a
Linux HOSTNAME 4.18.0-10-generic #11-Ubuntu SMP Thu Oct 11 15:13:55 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$

Allow me to explain my problem with a different perspective:

When I apply the URL pointing to the .PAC file into the "Network Proxy" of MATE, the web works fine with firefox.

However this setting is not transposed to terminal/CLI and even though browsing works in GUI it fails to work in CLI.

Any ideas why a UM system can browse in GUI and not in terminal?