According to users' different requirements for log search latency and log processing capabilities, CLS provides two storage classes: STANDARD and STANDARD_IA.
Basic Concepts
Storage classes: STANDARD storage and STANDARD_IA storage. These two storage classes differ in their storage media, features, performance, and pricing.
Log transition: a method for hierarchical log storage. Store newly generated data that requires SQL analysis and has high timeliness requirements in STANDARD storage. Store historical data that does not require SQL analysis and has low timeliness requirements in STANDARD_IA storage to reduce the cost of long-term log storage. For details, see Log Tiering. Features
|
| | |
| | |
| | |
| ✓ (2-second response time for 100 million data records) | ✓ (0.5-second response time for 100 million data records) |
| | |
| | |
| | |
| | |
| | |
Shipping to Cloud Object Storage (COS) | | |
| | |
Shipping to Elasticsearch (ES) | | |
Shipping to Serverless Cloud Function (SCF) | | |
| | |
| | |
Advantages
STANDARD
STANDARD storage is suitable for users who require statistical analysis and provides log search within seconds, real-time statistical analysis, real-time monitoring, streaming consumption, and other application capabilities.
Ops Monitoring and Troubleshooting: It performs real-time diagnosis of online issues, leveraging the sub-second log search capability to quickly search log content scattered across multiple machines, pinpoint the root cause of failures, and facilitate repairs. It also calculates various quality metrics in real time based on logs and triggers an alarm immediately when a metric exceeds its threshold, enabling developers and Ops personnel to promptly identify and resolve issues.
Streaming processing: Collects the tracking log data of petabyte scale scattered on multiple machines and streams the data to the user-built big data processing cluster in real time for subsequent data lake computing, for example, for the model data calculation business of a recommendation system.
STANDARD_IA
STANDARD_IA is suitable for infrequently accessed logs that do not require statistical analysis, such as archived audit logs. It provides the full-text log search capability, meeting users' requirements for backtracking and archiving historical logs. The overall usage costs of STANDARD_IA storage are 80% lower than those of STANDARD storage.
Historical logs: The explosive growth of log data makes it expensive to store and analyze logs on a large scale over months or even years. This can cause users to delete valuable data and miss out on important insights that long-term data can yield. STANDARD_IA can meet the needs of users to conduct large-scale statistical analysis and backtracking of historical data with low costs.
Non-critical business logs: During troubleshooting, developers need to pay more attention to ERROR and WARN logs and monitor them and generate alarms when necessary. Non-critical business logs, such as INFO logs, are only archived and need to be searched and analyzed in specific scenarios. Common users do not have specific requirements on the search latency of these logs. Using STANDARD_IA to store non-critical business logs can significantly reduce user costs and meet users' demands for infrequent search.
Audit logs: Logs for operation and security audits are collected to STANDARD_IA, and access behaviors, such as operation records of an account or an object, are analyzed via CLS's infrequently accessed log search capability to determine whether there are illegal operations. In addition, logs can be stored for more than 180 days to meet compliance audit requirements.
Operation Steps
2. In the left sidebar, select Log Topic, go to the Log Topic List, and click Create Log Topic.
Specifications and Limits