Libc-Database 本地搭建实战
2020-12-06 18:58:00 Author: mp.weixin.qq.com(查看原文) 阅读量:40 收藏

本文为看雪论优秀文章

看雪论坛作者ID:勇士小蓝

0x01 前言


我们以Ubuntu 16.04虚拟机为例讲述搭建方法,为防止环境冲突,请选择使用一台干净的虚拟机,若使用Ubuntu Server 16.04,可能会出现系统安装失败情况,若出现,请参考底部的参考链接。
 
另外,此教程中涉及到了部分文件的修改,我也对原项目进行了部分本地化,因此也可以直接获取我fork修改的项目进行搭建。
项目地址:
https://github.com/hebtuerror404/libc-database。欢迎Star~
 
本文中最后生成的配置结束的PVM虚拟机已上传至百度云盘,需要的自取即可:
链接: 
https://pan.baidu.com/s/1mfG_ES4GzBrLyoGRerZDoA  
密码:9vpp

0x02 安装环境所需的依赖

1. 更新系统内的所有程序包(需要较长时间)

使用命令sudo apt-get update && sudo apt-get upgrade -y
 
注意:可以使用以下命令换源以加速下载
sudo cp /etc/apt/sources.list /etc/apt/sources.list.baksudo vi /etc/apt/sources.list# vi 内运行以下两条命令:% s/us.archive.ubuntu.com/mirrors.aliyun.com/g:wq

2. 通过apt安装必要软件包

使用命令
sudo apt-get install git vim wget curl python python3 python-pip python3-pip binutils file wget rpm2cpio cpio zstd nginx libffi6 libffi-dev software-properties-common -ysudo add-apt-repository ppa:deadsnakes/ppasudo apt-get updatesudo apt-get install python3.6sudo curl https://bootstrap.pypa.io/get-pip.py -o /get-pip.pysudo python3.6 /get-pip.pysudo ln -s /usr/local/bin/pip /usr/bin/pip

3. 通过pip安装必要软件包

使用命令
sudo pip install elasticsearch

4. 安装 docker 以及 docker-compose

此处推荐使用以下命令直接获取docker
curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun

docker成功安装后可以去阿里云官网获取镜像加速器,然后按照官网所述直接做如下配置即可:
sudo mkdir -p /etc/dockersudo tee /etc/docker/daemon.json <<-'EOF'{  "registry-mirrors": ["https://<镜像加速器ID>.mirror.aliyuncs.com"]}EOFsudo systemctl daemon-reloadsudo systemctl restart docker

然后运行sudo pip install docker-compose -i https://mirrors.aliyun.com/pypi/simple/安装docker-compose
 
注意:此处安装完成后,可能产生以下错误:
此时可以使用which docker-compose来确定核心文件位置,然后建立符号链接即可。

5. 安装nodejs以及npm

首先运行以下命令获取nodejs:
curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -sudo apt-get install -y nodejs

然后安装npm:
sudo npm install -g nsudo n latest

0x03 获取 libc-database 项目并处理


1. 获取项目

使用命令git clone https://github.com/niklasb/libc-database.git获取此项目。

2. 修改项目内的Node.js源码

修改libc-database/searchengine/frontend/src/App.js。
import React, { useState, useEffect, useCallback, useRef } from 'react';import './App.css';import 'fontsource-roboto';import Button from '@material-ui/core/Button';import Grid from '@material-ui/core/Grid';import TextField from '@material-ui/core/TextField';import Link from '@material-ui/core/Link';import CircularProgress from '@material-ui/core/CircularProgress';import Table from '@material-ui/core/Table';import TableBody from '@material-ui/core/TableBody';import TableCell from '@material-ui/core/TableCell';import TableRow from '@material-ui/core/TableRow'; import { makeStyles } from '@material-ui/core/styles';  - const API_BASE = 'https://libc.rip/api';+ const API_BASE = 'http://127.0.0.1/api'; const api = async (path, data) => {  let resp = await fetch(`${API_BASE}${path}`, {    method: 'POST',    mode: 'cors',    cache: 'no-cache',    headers: {      'Content-Type': 'application/json'    },    body: JSON.stringify(data),  });  return await resp.json();}; const useStyles = makeStyles((theme) => ({  root: {    '& .MuiTextField-root': {      margin: theme.spacing(1),    },    '& .MuiButton-root': {      margin: theme.spacing(1),    },    '& .remove': {      marginTop: '1.2rem',      height: '2rem',    },    '& .findbutton': {      marginTop: '1.2rem',    },  },  table: {    marginTop: '1rem',    marginBottom: '1rem',  }}));  function SearchRow({ onChange = () => {}, onRemove = () => {} }) {  const [symbol, setSymbol] = useState("");  const [address, setAddress] = useState("");  const [addressValid, setAddressValid] = useState(true);   const onSymbolChange = useCallback((evt) => {    setSymbol(evt.target.value);  }, []);   const onAddressChange = useCallback((evt) => {    setAddress(evt.target.value);  }, []);   useEffect(() => {    const valid = !!address.match(/^(0x)?[0-9a-fA-F]*$/);    setAddressValid(valid);    onChange({valid, symbol, address});  }, [address, symbol, onChange]);   return (    <div>      <TextField label="Symbol name" value={symbol} onChange={onSymbolChange} />      <TextField label="Address" error={!addressValid} value={address} onChange={onAddressChange} />      <Button className="remove" variant="contained" color="secondary" onClick={onRemove}>        Remove      </Button>    </div>  );} function SearchForm({ onSearch = () => {} }) {  const classes = useStyles();  const [nextId, setNextId] = useState(0);  const [rows, setRows] = useState([]);  const [states, setStates] = useState({});  const onRemoveRef = useRef();  const onChangeRef = useRef();   const makeRow = (id) => {    return (      <SearchRow key={id}        onRemove={() => onRemoveRef.current(id)}        onChange={(obj) => onChangeRef.current(id, obj)} />);  };   const isEmpty = useCallback((i) => {    let state = states[rows[i].key];    return !state || (!state.symbol && !state.address);  }, [rows, states]);   // Add new empty rows automatically  useEffect(() => {    let need = true;    for (let i = 0; i < rows.length; ++i) {      if (isEmpty(i)) {        need = false;        break;      }    }     if (need) {      setRows(rows => rows.concat([makeRow('' + nextId)]));      setNextId(id => id + 1);    }  }, [rows, states, nextId, isEmpty]);   // Remove superfluous rows at the end  useEffect(() => {    let i = rows.length - 1;    while (i >= 1 && isEmpty(i) && isEmpty(i-1)) {      --i;    }    if (i < rows.length - 1) {      setRows(rows => rows.slice(0, i+1));    }  }, [rows, states, nextId, isEmpty]);   const onRemove = useCallback((id) => {    for (let i = 0; i < rows.length; ++i) {      if (rows[i].key === id) {        setRows(rows.slice(0, i).concat(rows.slice(i+1)));        return;      }    }  }, [rows]);   const onChange = useCallback((id, obj) => {    setStates({...states, [id]: obj});  }, [states]);   onChangeRef.current = onChange;  onRemoveRef.current = onRemove;   const onSubmit = useCallback(() => {    let symbols = {};    for (let row of rows) {      let state = states[row.key];      if (state && state.valid && state.address && state.symbol) {        symbols[state.symbol] = state.address;      }    }    onSearch({"symbols": symbols});  }, [rows, states, onSearch]);   const isValid = useCallback(() => {    let cnt = 0;    for (let row of rows) {      let state = states[row.key];      if (!state)        continue;      if (!state.valid)        return false;      if (state.address && state.symbol)        cnt++;    }    return cnt > 0;  }, [rows, states]);   return (    <form className={classes.root}>      {rows}       <div>        <Button          disabled={!isValid()}          variant="contained"          className="findbutton"          color="primary"          onClick={onSubmit}>          Find        </Button>      </div>    </form>  );} function Result({ id, buildid, md5, symbols, download_url }) {  const classes = useStyles();  const [open, setOpen] = useState(false);   const onToggle = useCallback((evt) => {    evt.preventDefault();    setOpen(!open);  }, [open]);   let symbolRows = Object.entries(symbols).map(([k, v]) => (    <TableRow key={k}>      <TableCell><code>{k}</code></TableCell>      <TableCell><code>{v}</code></TableCell>    </TableRow>  ));   return (    <div>      <Link href='#' onClick={onToggle}>{id}</Link>      {open && (        <Table size="small" className={classes.table}>          <TableBody>            <TableRow>              <TableCell>Download</TableCell>              <TableCell>                <Link href={download_url} download>Click to download</Link>              </TableCell>            </TableRow>            <TableRow>              <TableCell>BuildID</TableCell>              <TableCell>{buildid}</TableCell>            </TableRow>            <TableRow>              <TableCell>MD5</TableCell>              <TableCell>{md5}</TableCell>            </TableRow>            {symbolRows}          </TableBody>        </Table>      )}    </div>  );} function App() {  const [loading, setLoading] = useState(false);  const [results, setResults] = useState(null);   const onSearch = (data) => {    setLoading(true);    (async () => {      try {        setResults(await api('/find', data));      } finally {        setLoading(false);      }    })();  };   return (    <div className="App">      <p>Powered by the <Link href="https://github.com/niklasb/libc-database/tree/master/searchengine">libc-database search API</Link></p>       <Grid container spacing={2}>        <Grid item sm={12} md={6}>          <h3>Search</h3>          <SearchForm onSearch={onSearch} />        </Grid>        <Grid item sm={12} md={6}>          <h3>Results</h3>          {loading && <CircularProgress />}          {results !== null && results.map(x => <Result {...x} />)}        </Grid>      </Grid>    </div>  );} export default App;

3. 生成此项目

接下来进入项目文件夹并生成Web端主项目:
cd libc-database/searchengine/frontend/sudo npm installsudo npm run build

注意:若npm安装过慢,可以尝试使用cnpm,官网对于cnpm的描述如下:
(1)这是一个完整 npmjs.org 镜像,你可以用此代替官方版本(只读),同步频率目前为 10分钟 一次以保证尽量与官方服务同步。
(2)从 registry.npm.taobao.org 安装所有模块。当安装的时候发现安装的模块还没有同步过来,淘宝 NPM 会自动在后台进行同步。并且会让你从官方 NPM registry.npmjs.org 进行安装。下次你再安装这个模块的时候。就会直接从 淘宝 NPM 安装了。
要使用cnpm则运行以下命令:
npm install -g cnpm --registry=https://registry.npm.taobao.orgcd libc-database/searchengine/frontend/sudo cnpm installcnpm run build

0x04 本地获取所有libc

注意:若可以在虚拟机内部开启VPN以加速访问境外网站,请在此步开启。
 
使用以下命令获取/更新所有libc文件到本地,此步骤依据网络连接状况需要耗时30分钟 ~ 2小时,请耐心等待。
cd libc-database/./get ubuntu debian centos kali parrotsec./get arch./get rpm

./get arch命令可能极其缓慢,属于正常情况,请耐心等待~
 
./get rpm命令可能会因cpio错误而退出,属于正常情况,直接无视即可~

0x05 本地启动查询服务


1. 修改源文件

首先需要修改一系列文件,使之满足启动条件。
 
修改libc-database/searchengine/nginx.conf
user www-data;worker_processes auto;pid /run/nginx.pid;include /etc/nginx/modules-enabled/*.conf; events {  worker_connections 768;} http {  sendfile on;  tcp_nopush on;  tcp_nodelay on;  keepalive_timeout 65;  types_hash_max_size 2048;   include /etc/nginx/mime.types;  default_type application/octet-stream; -  ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE-  ssl_prefer_server_ciphers on;   access_log /var/log/nginx/access.log;  error_log /var/log/nginx/error.log;   gzip on;   server {-    listen 443;+    listen 80;    server_name libc.rocks;-    ssl on;-    ssl_certificate /etc/letsencrypt/live/libc.rip/fullchain.pem;-    ssl_certificate_key /etc/letsencrypt/live/libc.rip/privkey.pem;     location /api/ {      proxy_pass http://127.0.0.1:8000;      proxy_http_version 1.1;      proxy_set_header   Upgrade $http_upgrade;      proxy_set_header   Connection keep-alive;      proxy_set_header   Host $host;      proxy_cache_bypass $http_upgrade;    }     location /download/ {-      alias /home/niklas/libc-database/db/;+      alias /home/error404/libc-database/db/;    }     location / {-      alias /home/niklas/libc-database/searchengine/frontend/build/;+      alias /home/error404/libc-database/searchengine/frontend/build/;    }  }}
修改libc-database/searchengine/config.py
import os ES_INDEX_NAME = 'libcsearch' # ES_HOST = 'localhost'ES_HOST = 'es01' # DB_DIR = os.path.dirname(os.path.abspath(__file__)) + '/../db'DB_DIR = '/db' DEFAULT_SYMBOLS = [    '__libc_start_main_ret',    'system',    'dup2',    'str_bin_sh',    'read',    'write',    'puts',    'printf',]- DOWNLOAD_URL = 'https://libc.rip/download/{}.so'+ DOWNLOAD_URL = 'https://127.0.0.1/download/{}.so'

修改libc-database/searchengine/api.py
openapi: 3.0.3info:  version: 1.0.0  title: libc-database search engine  description: ''paths:  /find:    post:      operationId: app.find      tags:      - libcsearch      description: |-        Look up libc by various attributes      requestBody:        content:          application/json:            schema:              $ref: '#/components/schemas/Libc'        required: true      responses:        '200':          content:            application/json:              schema:                type: array                items:                  $ref: '#/components/schemas/Libc'          description: ''   /libc/{id}:    post:      operationId: app.dump      tags:      - libcsearch      parameters:      - in: path        name: id        schema:          type: string        required: true      description: |-        Dump libc symbols      requestBody:        content:          application/json:            schema:              $ref: '#/components/schemas/DumpRequest'        required: true      responses:        '200':          content:            application/json:              schema:                $ref: '#/components/schemas/Libc'          description: '' components:  schemas:    Libc:      type: object      properties:        md5:          type: string        sha1:          type: string        sha256:          type: string        buildid:          type: string        id:          type: string        symbols:          type: object          additionalProperties:            type: string            pattern: '^(0x)?[a-fA-F0-9]+$'        download_url:          type: string          format: url          readOnly: true     DumpRequest:      type: object      properties:        symbols:          type: array          items:            type: string  servers:-  - url: https://libc.rip/api+  - url: http://127.0.0.1/api

2. 启动elasticsearch查询服务以及查询后端uwsgi服务(守护服务)

使用命令sudo docker-compose up -d,启动后可以使用docker ps检查启动情况:

3. 建立elasticsearch查询服务索引

在/libc-database/searchengine下,执行python3.6 -m index ../db。
 
建立完成后,使用curl -XGET 127.0.0.1:9200/_cat/indices/来测试索引建立情况。

4. 启动本地Nginx服务

最后,在/libc-database/searchengine下使用/usr/sbin/nginx -c nginx.conf启动本地Ngnix服务。

5. 测试服务状态

使用命令curl -X POST -H 'Content-Type: application/json' --data '{"symbols": {"strncpy": "db0", "strcat": "0x000000000d800"}}' http://127.0.0.1/api/find进行测试。
 

6. Enjoy It

断开虚拟机网络,以验证本地状态下的服务可用性。
 

0x06 查询服务的更新

执行下列命令即可
cd libc-database/./get ubuntu debian centos kali parrotsec./get arch./get rpmpython3.6 -m index ../db

0x07 参考链接

【原】光盘安装ubuntu出现BusyBox不能继续安装的终极解决方法
https://blog.csdn.net/cqinter/article/details/3555769

- End -

看雪ID:勇士小蓝

https://bbs.pediy.com/user-home-816358.htm

*本文由看雪论坛 勇士小蓝 原创,转载请注明来自看雪社区。

往期推荐

WannaMine4.0病毒主程序分析

病毒分析日志3_inline hook邮件窃密器

Linux内核入门--CVE-2017-1000112-UFO

CVE-2020-16898:Windows TCP/IP远程代码执行漏洞

公众号ID:ikanxue
官方微博:看雪安全
商务合作:[email protected]

球分享

球点赞

球在看

“阅读原文一起来充电吧!

文章来源: http://mp.weixin.qq.com/s?__biz=MjM5NTc2MDYxMw==&mid=2458377830&idx=1&sn=542284b1549279ef886bf3ba4a546091&chksm=b180eeec86f767faad6277ba72d613778afc257ae6b052263001fdc3a098f55c557f777efc3a#rd
如有侵权请联系:admin#unsafe.sh