使用Spring ai和Ollama完全私有化使用大模型

Tags: functions, ollama, 大模型

你已经安装ollama，假如有什么问题，搜索一下，非常方便，支持window，linux，macos。

执行 ollama pull llama3.1 ，下载llama3.1，登上一点时间，毕竟有好几个G需要下载。

示例说明：本例子中，我会通过创建一个加法，乘法工具，通过llama3.1模型分析文本，自动调用需要工具，在ollama中叫function，openai中的叫工具。

下面，直接上代码

1. 代码

package springai;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Description;

import java.util.function.Function;

@SpringBootApplication
public class SpringOllama implements CommandLineRunner {
    public static void main(String[] args) {
        SpringApplication.run(SpringOllama.class);
    }

    @Autowired
    ChatModel chatModel;

    public static class MockCalculate {
        public record Request(Double n1, Double n2) {
        }

        public record Response(Double r) {
        }
    }

    public static class MockCalculateAdd implements Function<MockCalculate.Request, MockCalculate.Response> {

        public MockCalculate.Response apply(MockCalculate.Request request) {
            System.out.println("add request: " + request);
            return new MockCalculate.Response(request.n1 + request.n2);
        }
    }

    public static class MockCalculateMultiply implements Function<MockCalculate.Request, MockCalculate.Response> {

        public MockCalculate.Response apply(MockCalculate.Request request) {
            System.out.println("mul request: " + request);
            return new MockCalculate.Response(request.n1 * request.n2);
        }
    }

    public static class MockCalculateDivide implements Function<MockCalculate.Request, MockCalculate.Response> {

        public MockCalculate.Response apply(MockCalculate.Request request) {
            System.out.println("div request: " + request);
            return new MockCalculate.Response(request.n1 / request.n2);
        }
    }

    @Bean
    @Description("calculate sum")
    public Function<MockCalculate.Request, MockCalculate.Response> add() {
        return new MockCalculateAdd();
    }

    @Bean
    @Description("calculate multiplication")
    public Function<MockCalculate.Request, MockCalculate.Response> multiply() {
        return new MockCalculateMultiply();
    }

    @Bean
    @Description("calculate divide")
    public Function<MockCalculate.Request, MockCalculate.Response> divide() {
        return new MockCalculateDivide();
    }

    @Override
    public void run(String... args) {
        ChatClient.Builder builder = ChatClient.builder(chatModel);
        ChatClient chatClient = builder.build();
        ChatClient.CallResponseSpec responseSpec = chatClient.prompt()
                .system("you are a calculate, you need analyze the user input, and solve it step by step, use the tools to calculate the value.")
                .user("hello, what is 30 increase 47%")
                .functions("add", "multiply", "divide")
                .call();
        System.out.println(responseSpec.content());
        System.exit(0);
    }
}

1. 项目文件

plugins {
	id 'java'
	id 'org.springframework.boot' version '3.2.4'
	id 'io.spring.dependency-management' version '1.1.4'
}

group = 'org.bk.langchain4j'
version = '0.0.1-SNAPSHOT'

java {
	sourceCompatibility = '21'
}

ext {
	langchain4jVersion = "0.32.0"
}

repositories {
	mavenCentral()
	maven { url 'https://repo.spring.io/milestone' }
	maven { url 'https://repo.spring.io/snapshot' }
}

dependencies {
	implementation 'org.springframework.boot:spring-boot-starter-web'
	implementation 'org.springframework.session:spring-session-core'
	implementation 'org.springframework.ai:spring-ai-ollama'
	implementation 'org.springframework.ai:spring-ai-ollama-spring-boot-starter'
	testImplementation 'org.springframework.boot:spring-boot-starter-test'
	implementation platform("org.springframework.ai:spring-ai-bom:1.0.0-SNAPSHOT")
}

tasks.named('test') {
	useJUnitPlatform()
}

1. 配置文件

spring:
  ai:
    ollama:
      chat:
        options:
          model: "llama3.1"

1. 执行结果

add request: Request[n1=0.47, n2=30.0]
add request: Request[n1=14.1, n2=30.0]
To calculate the increase of 47% on 30, I will use the following steps:

1. First, I need to convert the percentage into a decimal by dividing it by 100: 47 / 100 = 0.47
2. Then, I will multiply the original value (30) by this decimal: 30 * 0.47 = 14.1
3. Finally, I will add this result to the original value: 30 + 14.1 = 44.1

Therefore, 30 increased by 47% is equal to 44.10.