selenium中的几个骚操作


写在前面

随着业务需求的增多,遇到的难题也越来越多了,所以想写篇文章记录一下
目前我手上没有代码,我一时半会儿也想不起来,等上班了再copy过来吧
先写一下解决了哪些问题

problems

1、控制chrome自动清空缓存

from selenium import webdriver
import time

browser = webdriver.Chrome()
browser.implicitly_wait(10)

# 访问这个地址会弹出清空缓存确认框
browser.get('chrome://settings/clearBrowserData')

time.sleep(2)

# 这串代码可以获取确认到确认按钮元素
Button = driver.execute_script("return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')")

# 点击确认按钮
Button.click()
time.sleep(2)

2、下载exe等类型的文件时,需确认保存

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

# 实例化一个Options
chromeoptions = Options()

# 用于定义下载不弹窗和默认下载地址
prefs = {"download.default_directory": "D:\download","download.prompt_for_download": False}
chromeoptions.add_experimental_option("prefs", prefs)

browser = webdriver.Chrome(chrome_options=chromeoptions)

# 这串代码发挥主要作用,但其中原理目前还没有搞清楚
browser.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': "D:\download"}}
command_result = browser.execute("send_command", params)

3、脚本打包成exe后,运行时出现cmd黑窗
说下这个的解决方案,使用ctypes调用WindowsAPI隐藏掉console窗口,最简单的方法是修改selenium的源码,但是这样把demo交给别人的时候,还要让别人去做修改,这样就不是很方便,所以重写相关类的某个方法,这样会好一些。
webdriver.py

import errno
import os
import platform
import subprocess
from subprocess import PIPE
import time
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.chrome.service import Service as ChromeService

import warnings
from selenium import webdriver
from selenium.webdriver.remote.webdriver import WebDriver as RemoteWebDriver
from selenium.webdriver.chrome.remote_connection import ChromeRemoteConnection

# 用于隐藏程序运行时的cmd窗口
from win32process import CREATE_NO_WINDOW

class Service(ChromeService): 
    # 重写start方法,为self.process添加creationflags参数
    def start(self):
        try:
            cmd = [self.path]
            cmd.extend(self.command_line_args())
            self.process = subprocess.Popen(cmd, env=self.env,
                                            close_fds=platform.system() != 'Windows',
                                            stdout=self.log_file,
                                            stderr=self.log_file,
                                            stdin=PIPE,
                                            creationflags=CREATE_NO_WINDOW)
        except TypeError:
            raise
        except OSError as err:
            if err.errno == errno.ENOENT:
                raise WebDriverException(
                    "'%s' executable needs to be in PATH. %s" % (
                        os.path.basename(self.path), self.start_error_message)
                )
            elif err.errno == errno.EACCES:
                raise WebDriverException(
                    "'%s' executable may have wrong permissions. %s" % (
                        os.path.basename(self.path), self.start_error_message)
                )
            else:
                raise
        except Exception as e:
            raise WebDriverException(
                "The executable %s needs to be available in the path. %s\n%s" %
                (os.path.basename(self.path), self.start_error_message, str(e)))
        count = 0
        while True:
            self.assert_process_still_running()
            if self.is_connectable():
                break
            count += 1
            time.sleep(1)
            if count == 30:
                raise WebDriverException("Can not connect to the Service %s" % self.path)

## 重写webdriver.Chrome中的init方法,实例化self.service时使用上面定义的类
class MyChrome(webdriver.Chrome):
    def __init__(self,executable_path="chromedriver", port=0,
                 options=None, service_args=None,
                 desired_capabilities=None, service_log_path=None,
                 chrome_options=None, keep_alive=True):
        if chrome_options:
            warnings.warn('use options instead of chrome_options',
                          DeprecationWarning, stacklevel=2)
            options = chrome_options

        if options is None:
            # desired_capabilities stays as passed in
            if desired_capabilities is None:
                desired_capabilities = self.create_options().to_capabilities()
        else:
            if desired_capabilities is None:
                desired_capabilities = options.to_capabilities()
            else:
                desired_capabilities.update(options.to_capabilities())

        self.service = Service(
            executable_path,
            port=port,
            service_args=service_args,
            log_path=service_log_path)
        self.service.start()

        try:
            RemoteWebDriver.__init__(
                self,
                command_executor=ChromeRemoteConnection(
                    remote_server_addr=self.service.service_url,
                    keep_alive=keep_alive),
                desired_capabilities=desired_capabilities)
        except Exception:
            self.quit()
            raise
        self._is_remote = False

保存上述文件,然后使用以下方式声明chromedriver

from webdriver import MyChrome
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
browser = MyChrome(options = chrome_options)

脚本打包成exe后不会再出现控制台窗口。
4、为chromedriver设置cookie 、

from selenium import webdriver
browser = webdriver.Chrome()
# 一定要先打开一次网站
browser.get('http://mp.weixin.qq.com')
Cookie = {
    "appmsglist_action_3925214187":"list",
    "mm_lang":"zh_CN"
    }
# 逐个设置cookie,name和value是指定的两个key
for key in Cookie.keys():
    cookie_dict = {
        'name':key,
        'value':Cookie[key]
        }
    browser.add_cookie(cookie_dict=cookie_dict)
# 再次打开网站,会发现已经登录成功
browser.get('http://mp.weixin.qq.com')

写在后面

就先写这三个吧,之前还有个擦除chromedriver指纹的,不过在另一篇文章有记录了
1和2是因为chrome更新换代,一些方法已经不再适用了,多搜一下还是能找到办法滴
至于3,主要是重写了类的某个方法,隐藏掉运行时的黑窗

2021-07-11更新

目前1和2写上去了,姿势均来自CSDN,下面放链接,3和4等等再写
Selenium 如何使用webdriver下载文件(chrome浏览器)
python + selenium清除Chrome缓存

2021-07-12更新

更新完毕~~