网络爬虫缺少安全证书的报错

2024-06-07 08:58

本文主要是介绍网络爬虫缺少安全证书的报错,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

解决 sun.security.validator.ValidatorException: PKIX path building failed

 

今天用java HttpClients写爬虫在访问某Https站点报如下错误:

 

sun.security.validator.ValidatorException: PKIX path building failed...

 

网上找了找解决方案,看到这篇博客:http://www.oschina.net/question/12_19249

http://www.oschina.net/code/snippet_12_4092

 

经测试,解决问题,这里做个记录。

 

复制代码
/** Copyright 2006 Sun Microsystems, Inc.  All Rights Reserved.** Redistribution and use in source and binary forms, with or without* modification, are permitted provided that the following conditions* are met:**   - Redistributions of source code must retain the above copyright*     notice, this list of conditions and the following disclaimer.**   - Redistributions in binary form must reproduce the above copyright*     notice, this list of conditions and the following disclaimer in the*     documentation and/or other materials provided with the distribution.**   - Neither the name of Sun Microsystems nor the names of its*     contributors may be used to endorse or promote products derived*     from this software without specific prior written permission.** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR* PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.*/import java.io.*;
import java.net.URL;import java.security.*;
import java.security.cert.*;import javax.net.ssl.*;public class InstallCert {public static void main(String[] args) throws Exception {String host;int port;char[] passphrase;if ((args.length == 1) || (args.length == 2)) {String[] c = args[0].split(":");host = c[0];port = (c.length == 1) ? 443 : Integer.parseInt(c[1]);String p = (args.length == 1) ? "changeit" : args[1];passphrase = p.toCharArray();} else {System.out.println("Usage: java InstallCert <host>[:port] [passphrase]");return;}File file = new File("jssecacerts");if (file.isFile() == false) {char SEP = File.separatorChar;File dir = new File(System.getProperty("java.home") + SEP+ "lib" + SEP + "security");file = new File(dir, "jssecacerts");if (file.isFile() == false) {file = new File(dir, "cacerts");}}System.out.println("Loading KeyStore " + file + "...");InputStream in = new FileInputStream(file);KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());ks.load(in, passphrase);in.close();SSLContext context = SSLContext.getInstance("TLS");TrustManagerFactory tmf =TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());tmf.init(ks);X509TrustManager defaultTrustManager = (X509TrustManager)tmf.getTrustManagers()[0];SavingTrustManager tm = new SavingTrustManager(defaultTrustManager);context.init(null, new TrustManager[] {tm}, null);SSLSocketFactory factory = context.getSocketFactory();System.out.println("Opening connection to " + host + ":" + port + "...");SSLSocket socket = (SSLSocket)factory.createSocket(host, port);socket.setSoTimeout(10000);try {System.out.println("Starting SSL handshake...");socket.startHandshake();socket.close();System.out.println();System.out.println("No errors, certificate is already trusted");} catch (SSLException e) {System.out.println();e.printStackTrace(System.out);}X509Certificate[] chain = tm.chain;if (chain == null) {System.out.println("Could not obtain server certificate chain");return;}BufferedReader reader =new BufferedReader(new InputStreamReader(System.in));System.out.println();System.out.println("Server sent " + chain.length + " certificate(s):");System.out.println();MessageDigest sha1 = MessageDigest.getInstance("SHA1");MessageDigest md5 = MessageDigest.getInstance("MD5");for (int i = 0; i < chain.length; i++) {X509Certificate cert = chain[i];System.out.println(" " + (i + 1) + " Subject " + cert.getSubjectDN());System.out.println("   Issuer  " + cert.getIssuerDN());sha1.update(cert.getEncoded());System.out.println("   sha1    " + toHexString(sha1.digest()));md5.update(cert.getEncoded());System.out.println("   md5     " + toHexString(md5.digest()));System.out.println();}System.out.println("Enter certificate to add to trusted keystore or 'q' to quit: [1]");String line = reader.readLine().trim();int k;try {k = (line.length() == 0) ? 0 : Integer.parseInt(line) - 1;} catch (NumberFormatException e) {System.out.println("KeyStore not changed");return;}X509Certificate cert = chain[k];String alias = host + "-" + (k + 1);ks.setCertificateEntry(alias, cert);OutputStream out = new FileOutputStream("jssecacerts");ks.store(out, passphrase);out.close();System.out.println();System.out.println(cert);System.out.println();System.out.println("Added certificate to keystore 'jssecacerts' using alias '"+ alias + "'");}private static final char[] HEXDIGITS = "0123456789abcdef".toCharArray();private static String toHexString(byte[] bytes) {StringBuilder sb = new StringBuilder(bytes.length * 3);for (int b : bytes) {b &= 0xff;sb.append(HEXDIGITS[b >> 4]);sb.append(HEXDIGITS[b & 15]);sb.append(' ');}return sb.toString();}private static class SavingTrustManager implements X509TrustManager {private final X509TrustManager tm;private X509Certificate[] chain;SavingTrustManager(X509TrustManager tm) {this.tm = tm;}public X509Certificate[] getAcceptedIssuers() {throw new UnsupportedOperationException();}public void checkClientTrusted(X509Certificate[] chain, String authType)throws CertificateException {throw new UnsupportedOperationException();}public void checkServerTrusted(X509Certificate[] chain, String authType)throws CertificateException {this.chain = chain;tm.checkServerTrusted(chain, authType);}}}
复制代码

编译:javac InstallCert.java 
运行:java InstallCert ecc.fedora.redhat.com

 

C:\Program Files\Java\jdk1.8.0_74\jre\lib\security

这篇关于网络爬虫缺少安全证书的报错的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/1038719

相关文章

解决idea启动项目报错java: OutOfMemoryError: insufficient memory

《解决idea启动项目报错java:OutOfMemoryError:insufficientmemory》:本文主要介绍解决idea启动项目报错java:OutOfMemoryError... 目录原因:解决:总结 原因:在Java中遇到OutOfMemoryError: insufficient me

Android使用java实现网络连通性检查详解

《Android使用java实现网络连通性检查详解》这篇文章主要为大家详细介绍了Android使用java实现网络连通性检查的相关知识,文中的示例代码讲解详细,感兴趣的小伙伴可以跟随小编一起学习一下... 目录NetCheck.Java(可直接拷贝)使用示例(Activity/Fragment 内)权限要求

Python多任务爬虫实现爬取图片和GDP数据

《Python多任务爬虫实现爬取图片和GDP数据》本文主要介绍了基于FastAPI开发Web站点的方法,包括搭建Web服务器、处理图片资源、实现多任务爬虫和数据可视化,同时,还简要介绍了Python爬... 目录一. 基于FastAPI之Web站点开发1. 基于FastAPI搭建Web服务器2. Web服务

Redis的安全机制详细介绍及配置方法

《Redis的安全机制详细介绍及配置方法》本文介绍Redis安全机制的配置方法,包括绑定IP地址、设置密码、保护模式、禁用危险命令、防火墙限制、TLS加密、客户端连接限制、最大内存使用和日志审计等,通... 目录1. 绑定 IP 地址2. 设置密码3. 保护模式4. 禁用危险命令5. 通过防火墙限制访问6.

idea突然报错Malformed \uxxxx encoding问题及解决

《idea突然报错Malformeduxxxxencoding问题及解决》Maven项目在切换Git分支时报错,提示project元素为描述符根元素,解决方法:删除Maven仓库中的resolv... 目www.chinasem.cn录问题解决方式总结问题idea 上的 maven China编程项目突然报错,是

Python爬虫HTTPS使用requests,httpx,aiohttp实战中的证书异步等问题

《Python爬虫HTTPS使用requests,httpx,aiohttp实战中的证书异步等问题》在爬虫工程里,“HTTPS”是绕不开的话题,HTTPS为传输加密提供保护,同时也给爬虫带来证书校验、... 目录一、核心问题与优先级检查(先问三件事)二、基础示例:requests 与证书处理三、高并发选型:

Java JUC并发集合详解之线程安全容器完全攻略

《JavaJUC并发集合详解之线程安全容器完全攻略》Java通过java.util.concurrent(JUC)包提供了一整套线程安全的并发容器,它们不仅是简单的同步包装,更是基于精妙并发算法构建... 目录一、为什么需要JUC并发集合?二、核心并发集合分类与详解三、选型指南:如何选择合适的并发容器?在多

Python实现简单封装网络请求的示例详解

《Python实现简单封装网络请求的示例详解》这篇文章主要为大家详细介绍了Python实现简单封装网络请求的相关知识,文中的示例代码讲解详细,感兴趣的小伙伴可以跟随小编一起学习一下... 目录安装依赖核心功能说明1. 类与方法概览2.NetHelper类初始化参数3.ApiResponse类属性与方法使用实

linux查找java项目日志查找报错信息方式

《linux查找java项目日志查找报错信息方式》日志查找定位步骤:进入项目,用tail-f实时跟踪日志,tail-n1000查看末尾1000行,grep搜索关键词或时间,vim内精准查找并高亮定位,... 目录日志查找定位在当前文件里找到报错消息总结日志查找定位1.cd 进入项目2.正常日志 和错误日

Debian 13升级后网络转发等功能异常怎么办? 并非错误而是管理机制变更

《Debian13升级后网络转发等功能异常怎么办?并非错误而是管理机制变更》很多朋友反馈,更新到Debian13后网络转发等功能异常,这并非BUG而是Debian13Trixie调整... 日前 Debian 13 Trixie 发布后已经有众多网友升级到新版本,只不过升级后发现某些功能存在异常,例如网络转